Blur correction camera system

ABSTRACT

A blur correction camera system includes a blur correction lens driven based upon the vibration detection signal detected by an angular velocity sensor, that corrects an image blur, a point-image function computing unit that computes a point spread function, and an image restoration computing unit that corrects an image blur by executing image restoration through image processing on a captured image by using the point spread function. The image blur that cannot be completely corrected by the blur correction lens is further corrected through image restoration so as to obtain a high quality image.

INCORPORATION BY REFERENCE

The disclosures of the following priority applications are hereinincorporated by reference:

-   Japanese Patent Application No. 2002-374644 filed Dec. 25, 2002-   Japanese Patent Application No. 2002-374661 filed Dec. 25, 2002-   Japanese Patent Application No. 2002-374687 filed Dec. 25, 2002-   Japanese Patent Application No. 2002-374704 filed Dec. 25, 2002-   Japanese Patent Application No. 2002-374724 filed Dec. 25, 2002-   Japanese Patent Application No. 2002-374748 filed Dec. 25, 2002-   Japanese Patent Application No. 2002-374739 filed Dec. 25, 2002-   Japanese Patent Application No. 2003-026098 filed Feb. 3, 2003

TECHNICAL FIELD

The present invention relates to a technology adopted in a camera or thelike to correct an image blur by detecting a vibration caused by anunsteady hand movement or the like.

BACKGROUND ART

There are cameras known in the related art, that have a blur correctingfunction for preventing an unsteady hand movement during a photographingoperation from lowering the quality of the captured image. Blurring iscorrected in such a camera by adopting one of the following two primarymethods.

The first method is an optical blur correction method in which avibration of the camera is detected by using a vibration detectionsensor such as an angular velocity sensor or an acceleration sensor anda blur is corrected by driving an optical system such as a photographiclens or a variable apex-angle prism in correspondence to the extent ofthe detected vibration (see, for instance, Japanese Laid Open PatentPublication No. S61-240780).

The second method is an electronic blur correction method in which theextent of blur is determined based upon the difference between thecaptured image and a previous image having been stored in memory on atemporary basis and the blur is corrected when reading out the image(see, for instance, Japanese Laid Open Patent Publication No.S63-187883). Through either of these two methods, the blur is correctedin real-time when the image is photographed.

There is another technology known in the related art which is used as analternative blur correction method to those described above, throughwhich a degraded image is restored as a blur-free image, unaffected byany unsteady hand movement. For instance, Japanese Laid Open PatentPublication No. S62-127976 discloses a method in which degradation of animage caused by a vibration occurring during the photographing operationis expressed as a point spread function and the image is restored as ablur-free image based upon the point spread function. There is also atechnology known in the related art adopted in conjunction with a cameraequipped with a vibration detection means alone, through which handmovement information is recorded and a blur is corrected by executingimage restoration processing based upon the information when reproducingthe image (see, for instance, Japanese Laid Open Patent Publication No.H 6-276512).

A specific method adopted in the image restoration processing is nowexplained. The term “image restoration” refers to a restoration of ablurred image, achieved by processing the blurred image based uponblur-related information so as to obtain an image manifesting a lesserextent of blurring.

With (x, y) representing positional coordinates on an image plane, o(x,y) representing an image obtained without experiencing any vibration(hereafter referred to as a raw image), z(x, y) representing an imagedegraded due to vibration (hereafter referred to as a blurred image) andp(x, y) representing information of a point image having become spreaddue to vibration (hereafter referred to as a point spread function),o(x, y), z(x, y) and p(x, y) achieve a relationship expressed asfollows;z(x,y)=o(x,y)*p(x, y,)  (1)In the expression above, “*” indicates a convolution (convolutedintegration) arithmetic operation, which is expressed specifically asfollows;z(x, y)=∫∫σ(x, y)p(x−x′, y−y′)dx′ dy′  (2)When the relationship is transformed into a relationship in a spatialfrequency (u, v) range through a Fourier transform, expressions (1) and(2) are rewritten as follows;Z(u, v)=O(u, v)·P(u, v)  (3)

Z(u, v), O(u, v) and P(u, v) respectively represent the spectrums ofz(x, y), o(x, y) and p(x, y). In addition, P(u, v) in expression (3) isspecifically referred to as a spatial frequency transfer function.

If the point-image function p(x, y) can be somehow ascertained inaddition to the blurred image z(x, y), their spectrums can be computedand then the spectrum O(u, v) of the raw image can be computed by usingthe following expression (4), which is a modified form of expression(3). $\begin{matrix}{{O\left( {u,v} \right)} = \frac{Z\left( {u,v} \right)}{P\left( {u,v} \right)}} & (4)\end{matrix}$

1/P(u, v) in expression (4) is specifically referred to as an inversefilter. The raw image o(x, y) can be determined through an inverseFourier transformation of the spectrum computed by using expression (4).FIGS. 6(a) to 6(c) and FIGS. 7(a) to 7(d) illustrate the imagerestoration executed in the related art.

In order to simplify the explanation, it is assumed that a uniform blurhas occurred along a single axis (the X axis), as shown in FIG. 6(b).

FIG. 7 (a) shows a section taken from the point spread function. Theresults of a Fourier transformation executed on this section in FIG.7(a), which are shown in FIG. 7(b), constitute the spatial frequencytransfer function of the blur shown in FIG. 6(a). This transfer functionhas characteristics of special interest in that it assumes the value 0at a plurality of points. The inverse filter of this function manifestsinstances of infinity, as shown in FIG. 7(c). When the inverse filter isincorporated in expression (4), the phenomenon expressed as in (5) belowoccurs with regard to a specific spatial frequency and, in such a case,the spectrum value of the raw image is indeterminate. $\begin{matrix}{{{O\left( {u,v} \right)} = {\frac{Z\left( {u,v} \right)}{P\left( {u,v} \right)} = {\frac{0}{0} = {indeterminate}}}}\quad} & (5)\end{matrix}$

When the transfer function indicates the value 0, there is a frequencycomponent that has not been transferred in the case of a blur(information has been lost), and accordingly, the expression aboveindicates that the lost frequency component cannot be restored. This, inturn, means that the complete recovery of the raw image is not possible.

It is to be noted that a Wiener filter expressed as below is actuallyused in the image restoration so as to ensure that the inverse filterdoes not manifest infinity. $\begin{matrix}{\frac{P^{*}\left( {u,v} \right)}{{{P\left( {u,v} \right)}}^{2} + {1/c}}C\text{:}\quad{constant}} & (6)\end{matrix}$

FIG. 7(d) is a graph of the Wiener filter.

The use of the Wiener filter ensures that O(u, v) is not allowed tobecome indeterminate, unlike in expression (5).

However, the following problems exist in the optical blur correction andthe image restoration in the related art described above.

(Problems of Optical Blur Correction)

An angular velocity sensor is normally used to detect vibration in theoptical blur correction. In order to convert the angular velocitydetected with the angular velocity sensor to an angle, the value(reference value) output from the sensor while it is in a resting stateduring the operation is needed. However, this reference value is knownto be readily affected by drift attributable to temperature changes.This issue is now explained in detail in reference to FIGS. 8(a) and8(b).

FIGS. 8(a) and 8(b) show the angular velocity sensor output containingthe drift component, reference value outputs and the extent of blurmanifesting on the image surface.

FIG. 8(a) shows the change occurring in the angular velocity sensoroutput value over time and in order to simplify the explanation, it isassumed that a vibration due to a hand movement, represented as a sinewave, has occurred. In FIG. 8(a), a waveform e0 indicates the vibrationsensor output when a vibration due to a hand movement represented as asine wave has occurred. In addition, waveforms e1 and e2 each representa reference value computed through a low pass filter, with the cutofffrequency in the waveform e1 set lower than in the waveform e2. Theoutput value in FIG. 8(a) indicates that the center of the vibrationshifts (drifts) as time elapses due to environment-related factors.

FIG. 8(b) shows the extents of blurs in the image surface manifestingafter executing blur correction based upon the angular velocity sensoroutput and the reference values in FIG. 8(a). Waveforms f0, f1 and f2 inFIG. 8(b) respectively correspond to the waveforms e0, e1 and e2 in FIG.8(a), with the waveform f0 representing the extent of blur manifestingin the image surface when no blur correction has been executed. Thewaveform f1 indicates that by using the reference value e1 with a lowercutoff frequency than that in the waveform f2, the high frequencycomponent is clipped more effectively but the extent of blur increasesover time. The waveform f2, on the other hand, indicates that while thedrift is reduced compared to that manifesting in the waveform f1 byusing the reference value with a higher cutoff frequency, thehigh-frequency component attributable to the hand movement cannot beeliminated.

As described above, the requirements that need to be satisfied toeliminate an image blur caused by an unsteady hand movement and therequirements that need to be satisfied to reduce the extent of theadverse effect of drift conflict with each other, and it is difficult toselect an optimal cutoff frequency for the low pass filter, at which theimage blur can be corrected to a desired extent and the effect of thedrift, too, is minimized. For this reason, a detection error is bound tomanifest in the detected vibration extent, which gives rise to a problemin that blurring is not completely eliminated from the image havingundergone the optical blur correction.

In addition, an optical blur correction apparatus often includes aswitch operated to switch on/off a blur correction operation, and if theuser fails to turn on the switch and a blur correction is not executedduring the photographing operation, a blurred image will result.

(Problems of Image Restoration)

Next, the problems of image restoration are explained.

It is known in the related art that the resolution of an image obtainedthrough restoration processing executed on a blurred image by using aWiener filter is improved over that of the raw image. However, since thefilter value is fairly large at a spatial frequency (u′, v′) at whichP(u′, v′)≈0, the noise component is amplified if the noise contained inthe image includes the spatial frequency component. This gives rise to aproblem in that the image quality is lowered by an unnecessary stripepattern that is bound to manifest in the image. While this stripepattern does not pose a very serious problem as long as the initialblurring is insignificant, it manifests prominently if the extent ofblurring is significant and in such a case, the stripe pattern becomesproblematic.

In addition, cameras having an image restoration processing function inthe related art are not capable of optically correcting blur but simplyrecord output data from a vibration sensor such as an angular velocitysensor and execute restoration processing based upon the vibrationinformation when reproducing the image. Thus, there is a problem in thatif an image blur occurs to a great extent, the image quality cannot beimproved through the image restoration processing due to the adverseeffect of the stripe pattern described above and the like.

Furthermore, while the point-image function needed in the imagerestoration processing is computed based upon information such as theangular velocity sensor output and an image restoration computation isexecuted based upon the results of the computation of the point-imagefunction, the volume of data output from the angular velocity sensor isextremely large, which necessitates lengthy arithmetic operations to beexecuted to result in poor computation efficiency. There is anotherproblem in that it requires a high-speed arithmetic processing unit.

Moreover, even when the data needed for the image restoration arerecorded into a recording medium or are transmitted to an externalrecipient without executing the point-image function computation or theimage restoration computation, the great volume of data requires alarge-capacity recording medium and a high-speed recording means or ahigh-speed communication means. Thus, the image restoration cannot berealized with ease and the implementation of the image restoration maylead to an increase in the cost.

DISCLOSURE OF THE INVENTION

The present invention adopts the following structures.

(1) A blur correction camera system according to the present inventionincludes a vibration detection unit that detects a vibration and outputsa vibration detection signal, a blur correction optical system that isdriven based upon the vibration detection signal and corrects an imageblur, an image-capturing unit that captures an image formed with aphotographic optical system that includes the blur correction opticalsystem and an image restoration computing unit that corrects an imageblur by executing image restoration through image processing on an imagecaptured by the image-capturing unit. It is desirable to further includea point spread function computing unit that computes a point spreadfunction and that the image restoration computing unit execute the imagerestoration by processing the image using the point spread function. Itis also desirable to further include a reference value computing unitthat computes a reference value for the vibration detection signal andthat the point spread function computing unit compute the point spreadfunction based upon calculation results of the reference value computingunit. The blur correction camera system should preferably include acamera that includes the vibration detection unit, the blur correctionoptical system, the image-capturing unit, the point spread functioncomputing unit, the reference value computing unit and an imagerecording unit that records an image, and an external device having theimage restoration computing unit, which is a device independent of thecamera and executes image restoration by using the image recorded by theimage recording unit and the point spread function input thereto.

A blur correction camera according to the present invention includes avibration detection unit that detects a vibration and outputs avibration detection signal, a blur correction optical system that isdriven based upon the vibration detection signal and corrects an imageblur, an image-capturing unit that captures an image formed by aphotographic optical system that includes the blur correction opticalsystem, an image recording unit that records the image captured by theimage-capturing unit and a point spread function computing unit thatcomputes a point spread function needed in an image restorationcomputation. It is desirable to further include a point spread functionoutput means for outputting the point spread function computed by thepoint spread function computing unit to the outside by utilizing theimage recording unit or a communication means. It is also desirable tofurther include a reference value computing unit that computes areference value for the vibration detection signal and that the pointspread function computing unit computes the point spread function basedupon calculation results of the reference value computing unit.

An image restoring device according to the present invention includes adata input unit that receives image data and a point spread functionobtained when capturing the image data through at least one ofcommunication with an outside and a medium and an image restorationcomputing unit that executes image restoration so as to correct an imageblur by executing image processing on the image data using the pointspread function.

A computer readable computer program product according to the presentinvention contains a blur correction control program, and the controlprogram includes a data input instruction for receiving image data and apoint spread function obtained when capturing the image data and animage restoration computation instruction for executing imagerestoration so as to correct an image blur by executing image processingon the image data using the point spread function. It is desirable thatthe computer program product be a recording medium on which the controlprogram is recorded. Alternatively, the computer program product may bea carrier wave on which the control program is embodied as a datasignal.

(2) A blur correction camera system according to the present inventionincludes a vibration detection unit that detects a vibration and outputsa vibration detection signal, a reference value computing unit thatcomputes a reference value for the vibration detection signal, a blurcorrection optical system that is driven based upon the vibrationdetection signal and the reference value and corrects an image blur, animage-capturing unit that captures an image formed with a photographicoptical system that includes the blur correction optical system, a pointspread function computing unit that computes a point spread function byusing the reference value or the vibration detection signal and an imagerestoration computing unit that corrects an image blur by executingimage restoration through image processing on the image captured by theimage-capturing unit using the point spread function. It is desirable tofurther include a point spread function computation switching unit thatselects one of the reference value and the vibration detection signal tobe used in the computation of the point spread function executed by thepoint spread function computing unit. The point spread functioncomputation switching unit may also function as a blur correctingoperation setting unit that switches ON/OFF a blur correcting operationby the blur correction optical system. It is desirable that when theblur correction optical system is to be engaged in the blur correctingoperation, the point spread function computing unit compute the pointspread function by using the reference value. If the blur correctionoptical system is not to be engaged in a blur correcting operation, thepoint spread function computing unit may compute the point spreadfunction by using the vibration detection signal. The blur correctioncamera system should preferably include a camera that includes thevibration detection unit, the blur correction optical system, theimage-capturing unit, the point spread function computing unit, thereference value computing unit and the image recording unit that recordsan image, and an external device having the image restoration computingunit, which is a device independent of the camera and executes imagerestoration by using the image recorded by the image recording unit andthe point spread function input thereto.

A blur correction camera according to the present invention includes avibration detection unit that detects a vibration and outputs avibration detection signal, a reference value computing unit thatcomputes a reference value for the vibration detection signal, a blurcorrection optical system that is driven based upon the vibrationdetection signal and the reference value and corrects an image blur, animage-capturing unit that captures an image formed by a photographicoptical system that includes the blur correction optical system, animage recording unit that records the image captured by theimage-capturing unit and a point spread function computing unit thatcomputes a point spread function using one of the reference value andthe vibration detection signal. It is desirable to further include apoint spread function output means for outputting the point spreadfunction computed by the point spread function computing unit to theoutside by utilizing one of the image recording unit and a communicationmeans.

(3) A blur correction camera according to the present invention includesa vibration detection unit that detects a vibration and outputs avibration detection signal, a reference value computing unit thatcomputes a reference value for the vibration detection signal a blurcorrection optical system that is driven based upon the reference valueand the vibration detection signal and corrects an image blur, animage-capturing unit that captures an image formed by a photographicoptical system that includes the blur correction optical system, a pointspread function computing unit that computes a point spread functionneeded in an image restoration computation based upon the referencevalue and an information volume reducing unit that reduces the volume ofinformation related to at least one of the reference value used in thecomputation of the point spread function and the computed point spreadfunction. It is desirable that the information volume reducing unitreduce the information volume by culling data related to at least one ofthe reference value and the computed point spread function. It is alsodesirable that the information volume reducing unit reduce theinformation volume by ensuring that there will still be a large enoughvolume of information required for the image restoration computation.

(4) A blur correction camera system according to the present inventionincludes a vibration detection unit that detects a vibration and outputsa vibration detection signal, an image-capturing unit that captures animage formed by a photographic optical system which includes a blurcorrection optical system as a raw image, and a raw image saving unitthat saves the raw image, an image restoration computing unit thatallows parameters related to image processing to be varied, executesimage restoration by executing image processing on the raw image usingthe parameter and creates a restored image obtained by correcting animage blur and a restoration result saving unit that saves at least oneof the parameters used in the image processing executed at the imagerestoration computing unit and the restored image in correspondence tothe raw image. It is desirable to further include a point spreadfunction computing unit that computes a point spread function, that theimage restoration computing unit execute the image restoration byprocessing the image using the point spread function and that theparameters include the point spread function. It is desirable that therestoration result saving unit be capable of saving at least one of aplurality of sets of parameters each corresponding to one of a pluralityof restored images and a plurality of restored images. The blurcorrection camera system should preferably include a camera thatincludes a vibration detection unit, the blur correction optical systemthat is driven based upon the vibration detection signal and corrects animage blur, the image-capturing unit, the point spread functioncomputing unit, a reference value computing unit that computes areference value for the vibration detection signal and the raw imagesaving unit, and an external device having the image restorationcomputing unit and a restoration result saving unit, which is a deviceindependent of the camera and executes image restoration by using a rawimage recorded at the raw image saving unit and the point spreadfunction input thereto.

An image restoring device according to the present invention includes adata input unit that receives raw image data and a point spread functionobtained when capturing the raw image data through at least one ofcommunication with an external device and a medium, an image restorationcomputing unit that allows parameters related to image processing to bevaried, executes image restoration through image processing on the rawimage data using parameters which include the point spread function andcreates a restored image obtained by correcting an image blur and arestoration result saving unit that saves at least one of the parametersused in the image processing executed by the image restoration computingunit and the restored image in correspondence to the raw image.

A computer readable computer program product according to the presentinvention contains a blur correction control program, and the controlprogram includes a data input instruction for receiving raw image dataand a point spread function obtained when capturing the raw image data,an image restoration computation instruction for creating a restoredimage by executing image restoration so as to correct an image blurthrough image processing executed on the raw image data using variableparameters related to the image processing, which include the pointspread function and a restoration result saving instruction for savingat least one of the parameters used in the image processing during theimage restoration computation step and the restored image incorrespondence to the raw image data. It is desirable that the computerprogram product be a recording medium on which the control program isrecorded. Alternatively, the computer program product may be a carrierwave on which the control program is embodied as a data signal.

(5) A blur correction camera according to the present invention includesa vibration detection unit that detects a vibration and outputs avibration detection signal, an optical blur correction means forcorrecting an image blur by driving a blur correction optical systembased upon the vibration detection signal, a point spread functioncomputing unit that computes a point spread function needed in imagerestoration in which the image blur is corrected through imageprocessing and an image restoration decision-making unit that makes adecision as to whether to enter an image restoration mode in which blurcorrection is executed through the image restoration or a preparatoryoperation for a blur correction to be achieved through the imagerestoration is executed. It is desirable that the image restorationdecision-making unit make a decision as to whether to enter the imagerestoration mode based upon the vibration detection signal. The imagerestoration decision-making unit may instead make a decision as towhether to enter the image restoration mode based upon a shutter speed.Alternatively, the image restoration decision-making unit may make adecision as to whether to enter the image restoration mode based uponthe focal length of the photographic optical system. The imagerestoration decision-making unit may make a decision as to whether toenter the image restoration mode based upon the point spread function aswell. A reporting means for reporting a decision made by the imagerestoration decision-making unit that the image restoration mode shouldnot be entered may be further provided. If the image restorationdecision-making unit determines that the image restoration mode shouldnot be entered, the image restoration mode does not need to be executed.When the image restoration decision-making unit determines that theimage restoration mode should not be entered, the point spread functiondoes not need to be saved.

(6) A blur correction camera according to the present invention includesa vibration detection unit that detects a vibration and outputs avibration detection signal, a reference value computing unit thatcomputes a reference value for the vibration detection signal, a blurcorrection optical system that is driven based upon the reference valueand the vibration detection signal and corrects an image blur, a drivecontrol unit that controls an operation of the blur correction opticalsystem based upon the vibration detection signal and the referencevalue, a point spread function computing unit that computes based uponthe reference value a point spread function needed in image restorationexecuted to correct image blur through image processing and a blurcorrection mode selection unit that selects whether to enter an imagerestoration mode in which blur correction is executed through the imagerestoration or a preparatory operation for blur correction to beachieved through image restoration is executed in addition to an opticalblur correcting operation executed by engaging the blur correctionoptical system in blur correction. The drive control unit modifies thecontents of control implemented on the blur correction optical system incorrespondence to the selection made by the blur correction modeselection unit. It is desirable that the drive control unit modify thecontents of the control implemented on the blur correction opticalsystem by adjusting a method for reference value computation incorrespondence to the selection made by the blur correction modeselection unit. The reference value computing unit may compute thereference value by using a low pass filter and the drive control unitmay modify the contents of the control implemented on the blurcorrection optical system by adjusting a cutoff frequency of the lowpass filter. If the selection made by the blur correction mode selectionunit indicates that image restoration is to be executed, the drivecontrol unit should set the cutoff frequency to a higher level than thecutoff frequency set when the selection made by the blur correction modeselection unit indicates that image restoration is not to be executed.

(7) A blur correction camera according to the present invention includesa vibration detection unit that detects a vibration and outputs avibration detection signal, an optical blur correction means forcorrecting an image blur by driving a blur correction optical systembased upon the vibration detection signal, a point spread functioncomputing unit that computes a point spread function needed in imagerestoration executed to correct through image processing a blur thatcannot be completely corrected by the optical blur correction means anda blur correction mode selection unit that selects an optical blurcorrection mode in which blur correction is executed by engaging theoptical blur correction means in operation and an image restoration modein which blur correction is executed through image restoration or apreparatory operation for blur correction to be achieved through imagerestoration is executed. When the blur correction mode selection unitselects the image restoration mode, it also selects the optical blurcorrection mode in conjunction.

A blur correction camera according to the present invention includes avibration detection unit that detects a vibration and outputs avibration detection signal, an optical blur correction means forcorrecting an image blur by driving a blur correction optical systembased upon the vibration detection signal, a point spread functioncomputing unit that computes a point spread function needed in imagerestoration executed to correct through image processing a blur thatcannot be completely corrected by the optical blur correction means anda blur correction mode selection unit that selects an optical blurcorrection mode in which a blur correction is executed by engaging theoptical blur correction means in operation and an image restoration modein which blur correction is executed through image restoration or apreparatory operation for blur correction to be achieved through imagerestoration is executed. The blur correction mode selection unit is notallowed to enter the image restoration mode unless the optical blurcorrection mode is also selected.

A blur correction camera according to the present invention includes avibration detection unit that detects a vibration and outputs avibration detection signal, an optical blur correction means forcorrecting an image blur by driving a blur correction optical systembased upon the vibration detection signal, a point spread functioncomputing unit that computes a point spread function needed in imagerestoration executed to correct through image processing a blur thatcannot be completely corrected by the optical blur correction means anda blur correction mode selection unit that selects an optical blurcorrection mode in which a blur correction is executed by engaging theoptical blur correction means in operation and an image restoration modein which blur correction is executed through image restoration or apreparatory operation for blur correction to be achieved through imagerestoration is executed. The blur correction mode selection unit issuesa warning if the image restoration mode alone is selected without alsoselecting the optical blur correction mode.

A blur correction camera according to the present invention includes avibration detection unit that detects a vibration and outputs avibration detection signal, an optical blur correction means forcorrecting an image blur by driving a blur correction optical systembased upon the vibration detection signal, and a point spread functioncomputing unit that computes a point spread function needed in imagerestoration executed to correct through image processing a blur thatcannot be completely corrected by the optical blur correction. The pointspread function computing unit is enabled to execute the computation ofthe point spread function as the optical blur correction means isengaged in operation.

(8) A blur correction camera system according to the present inventionincludes a blur correction optical system that corrects an image blur, avibration detection unit that detects a vibration and outputs avibration signal, a reference value computing unit that computes areference value for the vibration signal, a drive unit that drives theblur correction optical system, a position detection unit that detects aposition of the blur correction optical system and outputs a positionsignal, a control unit that controls drive of the blur correctionoptical system based upon the reference value, the vibration signal andthe position signal so as to correct a blur manifesting in a subjectimage due to the vibration, an image-capturing unit that captures animage formed by a photographic optical system which includes the blurcorrection optical system, a control position error output unit thatoutputs as a control position error a difference between a target driveposition for the drive of the blur correction optical system by thecontrol unit and an actual drive position of the blur correction opticalsystem output by the position detection unit, and an image restorationcomputing unit that corrects an image blur by executing imagerestoration on the image captured by the image-capturing unit throughimage processing in which the control position error is taken intoconsideration. It is desirable to further include a point spreadfunction computing unit that computes a point spread function needed inthe image restoration computation and a function correcting unit thatcorrects the point spread function by using the control position errorand that the image restoration computing unit execute the imagerestoration by executing the image processing using the point spreadfunction having been corrected by the function correcting unit.Alternatively, a point spread function computing unit may further beprovided, that computes a point spread function needed in the imagerestoration computation, the point spread function may be computed basedupon one of (a) the reference value and the control position error, (b)the vibration signal and the control position error, (c) the referencevalue, the vibration signal and the control position error and (d) thecontrol position error, and the image restoration computing unit mayexecute the image restoration by using the point spread function in theimage processing. The blur correction camera system should preferablyinclude a camera that includes at least the vibration function unit, theblur correction optical system, the image-capturing unit, the pointspread function computing unit, the reference value computing unit andthe image recording unit that records an image, and an external devicehaving at least the image restoration computing unit, which is a deviceindependent of the camera and executes the image restoration using theimage recorded by the image recording unit and the point spread functioninput thereto. Alternatively, the blur correction camera system shouldpreferably include a camera that includes at least the vibrationfunction unit, the blur correction optical system, the image-capturingunit, the reference value computing unit and an image recording unitthat records an image, and an external device having at least the pointspread function computing unit and the image restoration computing unit,which is a device independent of the camera and executes the imagerestoration using the image recorded by the image recording unit and thepoint spread function input thereto.

A blur correction camera according to the present invention includes ablur correction optical system that corrects an image blur, a vibrationdetection unit that detects a vibration and outputs a vibration signal,a reference value computing unit that computes a reference value for thevibration signal, a drive unit that drives the blur correction opticalsystem, a position detection unit that detects a position of the blurcorrection optical system and outputs a position signal, a control unitthat controls drive of the blur correction optical system based upon thereference value, the vibration signal and the position signal so as tocorrect a blur manifesting in a subject image due to the vibration, animage-capturing unit that captures an image formed by a photographicoptical system which includes the blur correction optical system, animage recording unit that records an image, a control position erroroutput unit that outputs as a control position error a differencebetween a target drive position for the drive of the blur correctionoptical system by the control unit and an actual drive position of theblur correction optical system output by the position detection unit, apoint spread function computing unit that computes a point spreadfunction needed in image restoration computation, a function correctingunit that corrects the point spread function by using the controlposition error and an external output means for outputting to anexternal device the point spread function having been corrected by thecorrecting unit via one of the image recording unit and a communicationmeans.

A blur correction camera according to the present invention includes ablur correction optical system that corrects an image blur, a vibrationdetection unit that detects a vibration and outputs a vibration signal,a reference value computing unit that computes a reference value for thevibration signal, a drive unit that drives the blur correction opticalsystem, a position detection unit that detects a position of the blurcorrection optical system and outputs a position signal, a control unitthat controls drive of the blur correction optical system based upon thereference value, the vibration signal and the position signal so as tocorrect a blur manifesting in a subject image due to the vibration, animage-capturing unit that captures an image formed by a photographicoptical system which includes the blur correction optical system, animage recording unit that records an image, a control position erroroutput unit that outputs as a control position error a differencebetween a target drive position for the drive of the blur correctionoptical system by the control unit and an actual drive position of theblur correction optical system output by the position detection unit, apoint spread function computing unit that computes a point spreadfunction needed in image restoration computation and an external outputmeans for outputting to an external device the point spread function viaone of the image recording unit and a communication means. In this blurcorrection camera, the point spread function is computed based upon oneof (a) the reference value and the control position error, (b) thevibration signal and the control position error, (c) the referencevalue, the vibration signal and the control position error and (d) thecontrol position error.

A blur correction camera according to the present invention includes ablur correction optical system that corrects an image blur, a vibrationdetection unit that detects a vibration and outputs a vibration signal,a reference value computing unit that computes a reference value for thevibration signal, a drive unit that drives the blur correction opticalsystem, a position detection unit that detects a position of the blurcorrection optical system and outputs a position signal, a control unitthat controls drive of the blur correction optical system based upon thereference value, the vibration signal and the position signal so as tocorrect a blur manifesting in a subject image due to the vibration, animage-capturing unit that captures an image formed by a photographicoptical system which includes the blur correction optical system, animage recording unit that records an image, a control position erroroutput unit that outputs as a control position error a differencebetween a target drive position for the drive of the blur correctionoptical system by the control unit and an actual drive position of theblur correction optical system output by the position detection unit,and an external output means for outputting to an external device thecontrol position error via one of the image recording unit and acommunication means.

An image restoring device according to the present invention includes adata input unit that receives through at least one of communication withan external device and a medium a control position error determinedbased upon a difference between a target drive position for a blurcorrection optical system and an actual drive position of the blurcorrection optical system output by a position detection unit, imagedata, a point spread function obtained when capturing the image data anda function correcting unit that corrects the point spread function byusing the control position error and an image restoration computing unitthat corrects an image blur by executing image restoration on the imagedata through image processing in which the point spread function havingbeen corrected by the function correcting unit is used.

An image restoring device according to the present invention includes adata input unit that receives through at least one of communication withan external device and a medium a control position error determinedbased upon a difference between a target drive position for a blurcorrection optical system and an actual drive position of the blurcorrection optical system output by a position detection unit, imagedata and a vibration signal obtained when capturing the image data, apoint spread function computing unit that computes a point spreadfunction needed in image restoration computation, a function correctingunit that corrects the point spread function by using the controlposition error and an image restoration computing unit that corrects animage blur by executing image restoration on the image data throughimage processing in which the point spread function having beencorrected by the function correcting unit is used.

An image restoring device according to the present invention includes adata input unit that receives through at least one of communication withan external device and a medium, at least one of a control positionerror determined based upon a difference between a target drive positionfor a blur correction optical system and an actual drive position of theblur correction optical system output by a position detection unit,image data and a vibration signal obtained when capturing the imagedata, a point spread function computing unit that computes a pointspread function needed in image restoration computation and an imagerestoration computing unit that corrects an image blur by executingimage restoration on the image data through image processing in whichthe point spread function is used. The point spread function is computedbased upon one of (a) a reference value determined based upon thevibration signal and the control position error, (b) the vibrationsignal and the control position error, (c) the reference value, thevibration signal and the control position error and (d) the controlposition error.

A computer readable computer program product according to the presentinvention contains a blur correction control program, and the controlprogram includes a data input instruction for receiving, through atleast one of communication with an external device and a medium, acontrol position error determined based upon a difference between atarget drive position for a blur correction optical system and an actualdrive position of the blur correction optical system output from aposition detection unit, image data and a point spread function obtainedwhen capturing the image data, a function correction instruction forcorrecting the point spread function by using the control position errorand an image restoration computation instruction for correcting an imageblur by executing image restoration on the image data through imageprocessing executed using the point spread function having beencorrected through a function correction step.

A computer readable computer program product according to the presentinvention contains a blur correction control program, and the controlprogram includes a data input instruction for receiving, through atleast one of communication with an external device and a medium, acontrol position error determined based upon a difference between atarget drive position for a blur correction optical system and an actualdrive position of the blur correction optical system output from aposition detection unit, image data and a vibration signal obtainedwhile capturing the image data, a point spread function computationinstruction for computing a point spread function needed in imagerestoration computation, a function correction instruction forcorrecting the point spread function by using the control position errorand an image restoration computation instruction for correcting an imageblur by executing image restoration on the image data through imageprocessing using the point spread function having been corrected througha function correction step.

A computer readable computer program product according to the presentinvention contains a blur correction control program, and the controlprogram includes a data input instruction for receiving, through atleast one of communication with an external device and a medium, atleast one of a control position error determined based upon a differencebetween a target drive position for a blur correction optical system andan actual drive position of the blur correction optical system outputfrom a position detection unit, image data and a vibration signalobtained while capturing the image data, a point spread functioncomputation instruction for computing a point spread function needed inimage restoration computation based upon one of (a) a reference valuedetermined based upon the vibration signal and the control positionerror, (b) the vibration signal and the control position error, (c) thereference value, the vibration signal and the control position error and(d) the control position error, and an image restoration computationinstruction for correcting an image blur by executing image restorationon the image data through image processing using the point spreadfunction. The computer program product is a recording medium on whichthe control program is recorded. Alternatively, it may be a carrier waveon which the control program is embodied as a data signal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the system configuration adopted in afirst embodiment of a blur correction camera according to the presentinvention;

FIG. 2 presents an example of a block configuration that may be adoptedin the blur correction camera in the first embodiment;

FIG. 3 is a control block diagram illustrating a control operationexecuted in a blur correction control unit in an optical correctionsystem;

FIGS. 4(a) to 4(c) illustrate the image restoration executed in theembodiment;

FIGS. 5(a) to 5(d) illustrate the image restoration executed in theembodiment;

FIGS. 6(a) to 6(c) area first set of figures illustrating imagerestoration executed in the related art;

FIGS. 7(a) to 7(d) are a second set of figures illustrating imagerestoration executed in the related art;

FIGS. 8(a) and 8(b) show an angular velocity sensor output containing adrift component, reference value outputs and an extent of blurringmanifesting in an image surface;

FIG. 9 is a block diagram showing a system configuration adopted in asecond embodiment of the blur correction camera according to the presentinvention;

FIG. 10 presents an example of a block configuration that maybe adoptedin the blur correction camera in the second embodiment;

FIG. 11 is a control block diagram illustrating a control operationexecuted in a blur correction control unit in the second embodiment;

FIG. 12 presents a flowchart of a camera operations executed in thesecond embodiment;

FIG. 13 presents a flowchart of exposure and image restoring operationsexecuted in the second embodiment;

FIG. 14 is a block diagram showing a system configuration adopted in athird embodiment of the blur correction camera according to the presentinvention;

FIG. 15 presents a flowchart of a basic operations of the camera thatare engaged in a blur correcting operation;

FIG. 16 presents a flowchart of a basic operation executed in the cameraset in an optical blur correcting operation mode in the thirdembodiment;

FIG. 17 presents a flowchart of a basic operation executed in the cameraset in an image restoring operation mode in the third embodiment;

FIG. 18 presents a detailed flowchart of an operation executed at animage restoration decision-making unit in the third embodiment to make adecision based upon vibration detection data as to whether or not apoint-image function computation is to be executed;

FIG. 19 is a detailed flowchart of an operation executed to obtainpoint-image function computation data;

FIG. 20 presents a flowchart of a basic operation of an imagereproduction device;

FIG. 21 presents a specific example of an image display and operationsthat may be executed with regard to various parameters;

FIGS. 22(a) to 22(c) each present a specific example of image displayand operations executed with regard to various parameters;

FIG. 23 schematically shows the relationship among restored images savedin S2110, parameters and a raw image;

FIG. 24 presents a flowchart of an image restoring operation executed inthe fourth embodiment;

FIG. 25 presents a detailed flowchart of an operation executed at theimage restoration decision-making unit in the fourth embodiment to makea decision based upon the point-image function as to whether or notimage restoration is to be executed;

FIG. 26 is a block diagram showing a system configuration adopted in afifth embodiment of the blur correction camera according to the presentinvention;

FIG. 27 presents a flowchart of operations executed at a camera body andan interchangeable lens during a photographing operation in the fifthembodiment; and

FIG. 28 presents a flowchart of operations executed at the camera bodyand the interchangeable lens during a photographing operation in thefifth embodiment, in continuation from FIG. 27.

BEST MODE FOR CARRYING OUT THE INVENTION

The following is a detailed explanation of the embodiments of thepresent invention, given in reference to the drawings and the like.

(First Embodiment)

FIG. 1 is a block diagram showing a system configuration adopted in afirst embodiment of a blur correction camera according to the presentinvention.

The blur correction camera 1 achieved in the embodiment constitutes acamera system having an optical blur correction function and is capableof image restoration. FIG. 2 is a diagram of a block configuration thatmay be adopted in the blur correction camera 1 realized as a single lensreflex camera that allows the use of interchangeable lens barrels. Asshown in FIG. 2, the camera system includes a camera body 101 and a lensbarrel 102.

The blur correction camera 1 is a digital still camera thatelectronically captures an image and includes an optical correctionsystem (or a blur correction optical system) 500.

Angular velocity sensors 10 in the optical system 500 constitute avibration detection unit that detects a vibration to which the blurcorrection camera 1 is subjected, as an angular velocity value. Theangular velocity sensors 10 detect the angular velocity by takingadvantage of the Coriolis force and output the results of the detectionas voltage signals.

The angular velocity sensors 10 and the processing of the signals outputfrom the angular velocity sensors are now explained in reference to FIG.2.

The angular velocity sensors 10 are each installed in correspondence toeither the X axis perpendicular to an optical axis I of a photographiclens or the Y axis perpendicular to the X axis so as totwo-dimensionally detect a vibration of the blur correction camera 1.The angular velocity sensors 10 are enabled to detect angular velocitiesonly while power is supplied thereto from a power supply unit 90. It isto be noted that while a single angular velocity sensor 10 is shown inFIG. 1 for simplification, FIG. 2 shows angular velocity sensors 10 aand 10 b corresponding to the X axis direction and the Y axis directionrespectively.

An amplification unit 420 amplifies the outputs from the angularvelocity sensors 10.

An A/D conversion unit 20, which is converters that convert analogsignals to digital signals, converts analog vibration signals outputfrom the angular velocity sensors 10 to digital signals and provides thedigital signals to blur correction control units 30.

An A/D converter 440 converts position information (analog signals)indicating the position of a blur correction lens 70, provided from aposition detection unit 60 to digital signals. Blur correction lensposition information resulting from the conversion is transmitted to ablur correction control unit 30.

A D/A converter 430 converts drive signals (digital signals) obtainedthrough computation executed at the blur correction control unit 30 toanalog signals. The analog signals resulting from the conversion aretransmitted to an optical drive unit 50.

Since one angular velocity sensor 10 normally provides a small output,the resolution of the angular velocity value is too low (the angularvelocity value per bit is excessively large) and an accurate vibrationdetection cannot be executed if the output is directly digitized at theA/D converter 20 and then is processed in a microcomputer 460. Thismeans that the accuracy of the blur correction is compromised.Accordingly, angular velocity signals are amplified at the amplificationunit 420 before they are input to the A/D converter 20. Since theresolution of the angular velocity values can be improved (the angularvelocity value per bit can be lowered) in the microcomputer 460 in thismanner, the accuracy of the blur correction is improved.

The amplification unit 420 includes two amplification units 420 a and420 b respectively corresponding to the angular velocity sensors 10 aand 10 b. Instead of using the amplification units simply for signalamplification, high frequency noise contained in the sensor outputs maybe reduced by disposing a low pass filter at each amplification unit aswell.

In addition, as shown in FIG. 2, the A/D conversion unit 20, the blurcorrection control unit 30, the optical drive unit 50, the positiondetection unit 60, the D/A converter 430 and the A/D converter 440includes respectively a pair of A/D conversion units 20 a and 20 b, apair of blur correction control units 30 a and 30 b, a pair of opticaldrive units 50 a and 50 b, a pair of position detection units 60 a and60 b, a pair of D/A converters 430 a and 430 b and a pair of A/Dconverters 440 a and 440 b are installed in correspondence to the X axisdirection and the Y axis direction. However, since the operationsexecuted along the X axis and the Y axis by the individual units areidentical, a collective explanation is provided below withoutdistinguishing the operation executed along the X axis from thatexecuted along the Y axis or vice versa.

The explanation is now given in reference to FIG. 1.

The blur correction control unit 30 obtains through computation a drivesignal to be used to drive the blur correction lens 70 by using avibration signal detected with the angular velocity sensor 10 andposition information indicating the position of the blur correction lens70 detected by the position detection unit 60 which is to be detailedlater and outputs the drive signal to the optical drive unit 50. Theblur correction control unit 30 also functions as a control positionerror output unit that outputs an error (control position error) asdetailed later.

The blur correction control unit 30 includes a reference value computingunit 31 (see FIG. 3). The reference value computing unit 31 computes areference value for the vibration signal provided by the angularvelocity sensor 10, and in this embodiment, it is achieved by using adigital low pass filter (LPF). The output of the LPF is used as thereference value. The control operation executed by the blur correctioncontrol unit 30 is to be described in detail later.

The optical drive unit 50 is an actuator that drives the blur correctionlens 70 based upon the drive signal output from the blur correctioncontrol unit 30.

The position detection unit 60 detects the position of the blurcorrection lens 70 along the X axis or the Y axis, based upon which theblur correction is executed. The output (position signal) from theposition detection unit 60 is transmitted to the blur correction controlunit 30 via the A/D converter 440.

The blur correction lens 70 is a blur correction optical system includedin the photographic optical system of the camera and constituted with asingle lens or a lens group made up of a plurality of lenses that isallowed to move within a plane substantially perpendicular to theoptical axis I of the photographic optical system. As the blurcorrection lens 70 is driven by the optical drive unit 50 along adirection substantially perpendicular to the optical axis I, the opticalaxis I of the photographic optical system becomes deflected.

A blur in an image such as a photograph occurs as the image in the imageforming plane moves during the exposure due to a vibration such as anunsteady hand movement to which the camera is subjected. In the blurcorrection camera 1 achieved in the embodiment, a vibration to which theblur correction camera 1 is subjected can be detected with the angularvelocity sensor 10. Once the vibration to which the blur correctioncamera 1 has been subjected is detected, the movement of the image onthe image forming plane caused by the vibration can be ascertained andthen the movement of the image on the image forming plane, i.e., theimage blur, can be corrected by driving the blur correction lens 70 soas to cancel out the movement of the image on the image forming plane.

In addition to the optical correction system 500 described above, theblur correction camera 1 includes a control unit 80, the power supplyunit 90, a point-image function computing unit 100, an image-capturingunit 110, an image recording unit 120, an exposure control unit 150, afocus lens position detection unit 160, a focal length detection unit170, an electronic flash control unit 180, an operation unit 190 and thelike.

The control unit 80 controls the overall operation of the blurcorrection camera 1 and executes various types of control computationand the like to control the blur correction control unit 30, thepoint-image function computing unit 100, the exposure control unit 150,the focus lens position detection unit 160, the focal length detectionunit 170, the electronic flash control unit 180 and the like.

The power supply unit 90 continuously supplies power to the componentsof the camera that require power, such as the angular velocity sensor10, as long as a halfway press timer 195 in FIG. 2 remains in an ONstate. When the halfway press timer is in an OFF state, the power supplystops. Accordingly, the detection of a camera vibration by the angularvelocity sensor 10 is enabled only while the halfway press timer 195 inthe camera is in an ON state.

The point-image function computing unit 100 is a point spread functioncomputing unit that computes a point-image function (point spreadfunction) effective during the exposure based upon various types ofinformation provided by the blur correction control unit 30, theexposure control unit 150, the focus lens position detection unit 160,the focal length detection unit 170 and the like.

While a point-image function converges on a single point if a perfectoptical blur correction is achieved with the blur correction lens 70alone, the optical blur correction is never complete in reality and forthis reason, a point-image function is distributed. In other words, theimage is bound to show image blur (blur correction residual error) thathas not been corrected through the use of the blur correction lens 70.The point-image function that is computed by the point-image functioncomputing unit is used when correcting through subsequent imageprocessing the blur correction residual error remaining on the imageforming plane, which has not been corrected with the blur correctionlens 70.

The image-capturing unit 110, which includes an image-capturing element111, an A/D conversion unit 112, a signal processing unit 113 and thelike, captures an image formed by the photographic optical system ontothe image forming plane and outputs image data to the image recordingunit 120. The image-capturing element 111 receives the subject imageformed on the image forming plane by the photographic optical system andconverts the subject image to image data constituted of analog signals.The A/D conversion unit 112 converts the analog image to a digitalimage. The signal processing unit 112 processes the image data havingbeen converted to digital signals by the A/D conversion unit 112.

In addition to images captured by the image-capturing unit 110,point-image functions computed by the point-image function computingunit 100, various types of information (parameters) required to executevarious types of image restoration processing and the like are recordedand saved in correspondence to the individual images at the imagerecording unit 120. The point-image functions, the various types ofinformation and the like may be recorded as headers embedded in theindividual image files or they may be directly embedded in the imagesthrough digital watermarking technology or the like. Alternatively, aseparate file may be created in correspondence to each image file andthe information may be written into the separate file.

In more specific terms, the image recording unit 120 may be achieved asa removable recording medium such as a compact disc (trademark) or asmart medium (trademark), or it may be achieved as a buffer memorycapable of image transfer.

The exposure control unit 150 controls the length of the exposure periodto elapse at the image-capturing element based upon the exposure timelength setting selected via a command dial (not shown) or the like.Exposure time length information and timing information indicating thetiming with which the exposure starts/ends are transmitted to thepoint-image function computing unit 100.

The focus lens position detection unit 160 detects the position of afocus lens (not shown). By detecting the position of the focus lens, thedistance from the image forming plane to the subject, which is neededwhen computing the point-image function, can be computed.

The focal point detection unit 170 detects the lens focal length f inthe photographic optical system during the photographing operation. Thelens focal length f, too, constitutes information needed to compute thepoint-image function.

The electronic flash control unit 180 controls the light emission at anelectronic flash unit 181.

The operation unit 190 includes a halfway press switch (SW1) 191 and afull press switch (SW2) 192.

The halfway press switch 191 enters an ON state by interlocking with ahalfway press operation of a shutter release button (not shown). As thehalfway press switch 191 enters an ON state, a photometric computationby a photometering unit (not shown), auto focus drive and the likestart. In addition, if the halfway press timer 195 has been in an OFFstate, it is switched to an ON state in synchronization as the halfwaypress switch 191 enters an ON state.

The full press switch 192 enters an ON state by interlocking with a fullpress operation of the shutter release button (not shown). As the fullpress switch 192 enters an ON state, a sequence of photographingoperations, which includes opening/closing of the shutter by a shuttermechanism (not shown) and an image acquisition by an image sensor, isexecuted.

An image restoration computing unit 210 executes image restorationprocessing to correct a blur contained in an image based upon image dataprovided by the image recording unit 120 in the blur correction camera1, the point-image function information which corresponds to the imagedata and the various parameters to be used in the image restorationprocessing. While the Wiener filter expressed in (6) is used in theimage restoration processing executed by the image restoration computingunit 210, the image restoration processing may be executed by adoptinganother method.

At an image display unit 220, an image photographed by a photographer oran image resulting from an image restoration is displayed, and a monitorunit of the camera is equivalent to the image display unit in theembodiment.

Next, the aspect of the embodiment related to the blur correctioncontrol unit 30, including the control implemented during the opticalblur correcting operation, is explained.

FIG. 3 is a control block diagram illustrating the control operationexecuted at the blur correction control unit 30 in the opticalcorrection system 500.

First, the angular velocity sensor 10 detects a vibration to which thecamera has been subjected. The angular velocity sensor 10 is normallyconstituted with a piezoelectric vibration angular velocity sensor thatdetects the Coriolis force. The output of the angular velocity sensor 10is input to the reference value computing unit (low-frequency componentextraction) 31 via the A/D converter 20.

The reference value computing unit 31 computes a vibration referencevalue based upon the output from the angular velocity sensor 10. Thereference value for a normal unsteady hand movement may be set to thevalue output while the angular velocity sensor 10 is in a completelystationary state (hereafter referred to as a zero output). However, thiszero output value fluctuates as environment conditions such as the driftand the temperature change and, for this reason, the reference valuecannot be set to a fixed value. Accordingly, the reference value needsto be computed based upon the actual operating state, i.e., based uponthe signal indicating the unsteady hand movement of the photographer, todetermine the zero output. A digital low pass filter (LPF) is used inthe reference value computation.

While it is desirable to set the cutoff frequency fc for the digital lowpass filter as low as possible, the adverse effect of the sensor drifttends to manifest more readily if the cutoff frequency fc is set too lowas has been explained in reference to the related art. However, thefrequency component equal to or lower than fc, which is not corrected inthe optical correction, manifests as a significant residual image blurif the cutoff frequency is set too high. Through the image restorationprocessing executed by using a point-image function obtained based uponthe reference value output which is not optically corrected, the imageblur remaining after the optical correction can be subsequentlyeliminated, as detailed later.

Following the reference value computation, a vibration detection signalobtained by subtracting the reference value from the vibration detectionsignal provided by the angular velocity sensor 10 is transmitted to anintegrating unit 32.

The integrating unit 32 executes time integration of the vibrationdetection signal expressed in units of angular velocity and thusconverts the vibration detection signal to a camera vibration angle. Itmay execute the time integration as expressed below in (7).θ(t)=θ(t−1)+C·(ω(t)−ω₀(t))  (7)

In expression (7), θ(t): target drive position, ω(t): vibrationdetection signal, ω0 (t): reference value and t: time (integer), with Crepresenting a constant determined in correspondence to conditions suchas the lens focal length.

The target drive position signal computed by the integrating unit 32 istransmitted to a target drive position computing unit 33.

At the target drive position computing unit 33, target drive positioninformation to be used when driving the blur correction lens 70 isobtained through computation executed by taking into consideration theinformation indicating the lens focal length f provided by the focalpoint detection unit 170 and the information indicating the subjectdistance D provided by the focus lens position detection unit 160 aswell as the vibration angle information having been transmitted from theintegrating unit 32.

Through PID control of the known art or the like, the blur correctioncontrol unit 30 ascertains the difference between a target driveposition information and the position information provided by theposition detection unit 60, which detects the position of the blurcorrection lens 70 so as to drive the blur correction lens 70, incorrespondence to the target drive position information and outputs adrive signal to be used to drive the optical system drive unit 50. Thedrive signal is provided to the optical system drive unit 50 via the D/Aconverter 430. As an electrical current is supplied to a coil at theoptical system drive unit 50 in response to the drive signal having beenthus output, the blur correction lens 70 can be driven along thedirection perpendicular to the optical axis.

The position detection unit 60 monitors the position of the blurcorrection lens 70 and based upon the lens position signal indicatingthe detected lens position, the blur correction control unit 30 executesfeedback control on the blur correction lens 70.

Next, the operation of the point-image function computing unit 100 isexplained.

The problem of blur correction by the optical correction system 500failing to completely correct the blur and some degree of residualblurring remaining in the image (the occurrence of blur correctionresidual error) has been discussed in the explanation of the related art(see FIGS. 8(a) and 8(b)). Such a blur correction residual error isprimarily attributable to the specific value assumed for the referencevalue. Accordingly, the point-image function computing unit 100 in theembodiment calculates a point-image function related to the blurcorrection residual error based upon the reference value.

The method adopted when computing the point-image function is brieflyexplained.

The point-image function is computed as expressed below. Namely, areference value computation average value ω0ave is subtracted from thereference value ω0 having been obtained, and then an error angle θ(t) isdetermined by integrating the difference. Then, a point spread functionX(t) of the image surface is determined based upon the focal lengthinformation f.ω₀ ave=Σω₀(t)/N  (8)θ′(t)=θ′(t−1)+C·(ω_(o)(t)−ω_(o) ave)  (9)X(t)=f·θ(t)  (10)

It is to be noted that if a teleconverter is mounted, it is necessary toadjust the focal length in correspondence to the magnification factor ofthe teleconverter. In addition, by correcting the point spread functionby using the subject distance information, the accuracy of the pointspread function is improved. In such a case, the following expression(11) may be used.X(t)=β·Rf·θ(t)  (11)

-   β: lateral magnification factor-   R: subject distance

The point spread function is obtained by executing the arithmeticoperations described above along the X direction and the Y direction andthen by expanding the arithmetic operation results over the X-Y plane.

It is to be noted that the method described above simply represents anexample of a point-image function computation, and the point-imagefunction may be computed through another method.

The point-image function thus computed is then transmitted to the imagerestoration computing unit 210. Based upon the point-image functiontransmitted thereto, the image restoration computing unit 210 executesimage restoration computation to correct the residual image blur thathas not been eliminated through the blur correcting operation of theblur correction lens 70 and thus obtains a high quality image from whichthe blur has been effectively eliminated.

In the image restoration processing in the related art, the vibrationdetection data indicating a vibration detected with an angular velocitysensor or the like are directly used to compute the point-image functionwhich is then used in the image restoration processing. However, thismethod is problematic in that when a significant extent of blurmanifests in the image, the image quality cannot be improved through theimage restoration processing. In contrast, the embodiment in which theblur is first corrected to some extent by engaging the optical blurcorrection mechanism in operation and then image restoration processingis executed by using the vibration information greatly improves theimage quality.

FIGS. 4(a) to 4(c) and FIGS. 5(a) to 5(d) illustrate the imagerestoration achieved in the embodiment.

In the embodiment, the image restoration is executed by using the imagedata and the blur information obtained after the blur correction by theoptical blur correction mechanism and thus, the extent of blur to becorrected through the image restoration is not excessive. This advantagebecomes obvious as the image restoration in the embodiment is comparedwith the image restoration shown in FIGS. 7(a) to 7(d). As the extent ofthe blur becomes greater, the frequency component which is nottransferred increases, making it more difficult to restore the image.The number of points at which the spatial frequency transfer functionassumes the value 0 in FIG. 5(b) is smaller than the number of points atwhich the function assumes the value 0 in FIG. 7(b). This indicates thatthe frequency component which is not transferred is reduced and thus theimage restoration can be executed with a higher level of effectiveness.

As described above in detail, the following advantages can be achievedin the first embodiment.

There are provided the blur correction optical system, which corrects animage blur, and the image restoration computing unit, which corrects animage blur through image restoration by executing image processing on animage captured by the image-capturing unit. They complement each otherwith the problems of the optical blur correction executed with the blurcorrection optical system addressed by the image restoration computingunit and the problems of the blur correction executed with imagerestoration addressed by the blur correction optical system. As aresult, a highly effective blur correction can be achieved.

The point spread function computing unit that computes a point spreadfunction is provided and the image restoration computing unit executesthe image restoration by processing the image using the point spreadfunction. Thus, by computing the point spread function and saving thepoint spread function during the photographing operation, the imagerestoration can be executed at any desired time point after thephotographing operation.

Since the point spread function computing unit computes the point spreadfunction based upon the results of the computation executed at thereference value computing unit, the residual blur which the blurcorrection by the blur correction optical system has failed to eliminatecan be expressed as the point spread function and, as a result, the blurthat has not been eliminated through the blur correction by the blurcorrection optical system can be corrected through the imagerestoration.

(Second Embodiment)

In this embodiment, the system configuration of the first embodiment ismodified by adding a blur correcting operation selector switch 194 and asignal flow control unit 452.

FIG. 9 is a block diagram showing the system configuration adopted inthe embodiment, and FIG. 10 is a diagram showing the block configurationadopted in the system. The same reference numerals are assigned tocomponents having functions similar to those in the first embodiment tominimize the need for a reputed explanation thereof.

FIG. 11 is a block diagram illustrating the operation executed at theblur correction control unit. FIG. 11 shows the signal flow control unit452 installed to switch on/off a blur correcting operation when theoutput from the reference value computing unit 31 is provided to thepoint-image function computing unit 100.

The blur correcting operation selector switch (VRSW) 194 is operated toswitch on/off an optical blur correcting operation. In addition, thesignal flow control unit 452 switches the data to be transmitted to thepoint-image function computing unit 100 in correspondence to the stateof the VRSW 194. It is to be noted that in a camera that allows the useof interchangeable lens barrels, the correcting operation selectorswitch 194 is disposed at the outer circumferential area of the lensbarrel 102 of the interchangeable lens, as shown in FIG. 10. Theoperation is executed as described below based upon the state of theselector switch 194 in the embodiment.

When the VRSW 194 is in an ON state, the lens correction lens 70 isdriven and an optical correcting operation and an operation that isrequired to implement image restoration are executed. Image restorationdata needed for the image restoration processing are transmitted fromthe reference value computing unit 31 to the point-image functioncomputing unit 100. If, on the other hand, the VRSW 194 is in an OFFstate, the blur correction lens 70 is not driven and no optical blurcorrecting operation is executed. The target drive position signaloutput from the target drive position computing unit 33 is transmittedto the point-image function computing unit 100 as the image restorationdata. It is to be noted that information indicating the current settingof the VRSW 194 is transmitted to the point-image function computingunit 100 regardless of whether it is in an ON state or an OFF state.

Next, the basic operation of the blur correction camera 1 executed inthe embodiment is explained.

FIG. 12 presents a flowchart of the basic operation of the cameraexecuted in the embodiment.

The following is an explanation of the camera operation executed in theembodiment, given in reference to the flowchart presented in FIG. 12.

It is to be noted that since the details of the operations executedalong the X direction and the Y direction are identical, the explanationis given without referring to a specific direction.

In step S1010, a decision is made as to whether or not the halfway pressswitch SW1 is in an ON state. The operation proceeds to step S1020 ifthe halfway press switch SW1 is currently in an ON state, whereas theoperation proceeds to step S1140 if the halfway press switch SW1 is inan OFF state.

In step S1020, a counter Tsw1 is reset to clear the count to 0. Thevalue of the count at the counter Tsw1, which measures the length oftime elapsing after the halfway press switch SW1 enters an OFF state isan integer. The counter holds the value 0 while the halfway press switchSW1 is in an ON state, and it engages in operation only while thehalfway press switch SW1 is in an OFF state and the halfway press timer195 is in an ON state.

In step S1030, a decision is made as to whether or not the halfway presstimer 195 is in an OFF state. The operation proceeds to step S1040 ifthe halfway press timer 195 is determined to be in an OFF state, whereasthe operation proceeds to step S1170 if the halfway press timer 195 isdetermined to be in an ON state.

In step S1040, a counter “t” is reset to clear the count to 0. Thecounter “t” measures the length of time over which the halfway presstimer 195 remains in an ON state. This counter is an integral valuecounter which starts a count operation immediately as the halfway presstimer 195 enters an ON state and continuously executes the countoperation as long as the halfway press timer 195 remains in an ON state.

In step S1050, the halfway press timer 195 is turned on. In step S1060,the angular velocity sensor 10 is turned on and a vibration detectionstarts. In addition, a conversion operation at the A/D converter 20,too, starts in this step. In step S1070, the computation of thereference value, which is executed based upon the output from theangular velocity sensor 10, starts. In step S1080, the computation of adrive signal to be used to drive the blur correction lens 70 starts. Instep S1090, a decision is made as to whether or not the VRSW 194 iscurrently in an ON state. If the VRSW 194 is determined to be on, theoperation proceeds to step S1100 to drive the blur correction lens 70.If, on the other hand, the VRSW 194 is currently set in an OFF state,the operation proceeds to step S1130.

In step S1100, drive of the blur correction lens 70 starts. In stepS1110, an exposure operation and an image restoring operation areexecuted. These operations are to be explained in detail later inreference to FIG. 13. In step S1120, the count at the counter “t” forthe halfway press timer 195 is incremented by one. In step S1130, thedrive of the blur correction lens 70 is stopped. It is to be noted thatif the drive of the blur correction lens 70 has already stopped prior tothis step, the stopped state is sustained.

In step S1140, a decision is made as to whether or not the halfway presstimer 195 is in an ON state. The operation proceeds to step S1150 if thehalfway press timer 195 is determined to be in an ON state, whereas theoperation returns to step S1010 if the halfway press timer 195 isdetermined to be in an OFF state to continuously check the halfway pressswitch SW1. At the time point at which the operation proceeds to stepS1150, the halfway press switch SW1 and the halfway press timer 195 atthe camera are respectively in an OFF state and an ON state. In order tomeasure the length of time elapsing while sustaining this condition, thecount at the counter Tsw1 is incremented by one.

In step S1160, a decision is made as to whether or not the count valueat the counter Tsw1 is smaller than a threshold value T_SW1. Thethreshold value T_SW1 is a constant used to determine the upper limitfor the counter Tsw1 and determines the length of time to elapse afterthe halfway press switch SW1 enters an OFF state until the halfway presstimer 195 shifts into an OFF state.

If the count at the counter Tsw1 is smaller than the threshold valueT_SW1, i.e., if an affirmative decision is made, the operation proceedsto step S1170 without turning off the halfway press timer 195. If, onthe other hand, the count at the counter Tsw1 is judged to be equal tothe threshold value T_SW1, i.e., if a negative decision is made in stepS1160, the operation proceeds to step S1220 to execute processing forturning off the halfway press timer 195 and processing that needs to beexecuted after the halfway press timer 195 is turned off.

In step S1170, the vibration detection is continuously executed bysustaining the angular velocity sensor 10 in an ON state. In addition,the conversion operation at the A/D converter 20, too, is continuouslyexecuted. In step S1180, the computation of the reference value iscontinuously executed. In step S1190, the computation of the drivesignal to be used to drive the blur correction lens 70 is continuouslyexecuted based upon the output from the angular velocity sensor 10 andthe reference value having been computed in step S1180.

In step S1200, a decision is made as to whether or not the VRSW 194 iscurrently in an ON state. If the VRSW 194 is judged to be set in an ONstate, the operation proceeds to step S1210 to continuously drive theblur correction lens 70. If, on the other hand, the VRSW 194 is set inan OFF state, the operation proceeds to step S1130.

In step S1210, the blur correction lens 70 is continuously driven. Instep S1220, the drive of the blur correction lens 70 is stopped. In stepS1230, the computation of the reference value is stopped. In step S1240,the power supply to the angular velocity sensor 10 stops, therebyturning off the angular velocity sensor 10. In step S1250, the halfwaypress timer 195 is turned off and then the operation returns to stepS1010 to detect the state of the halfway press switch (SW1) 191.

Next, the exposure operation and the image restoring operation executedin the camera achieved in the embodiment are explained.

FIG. 13 presents a detailed flowchart of the exposure operation and theimage restoring operation executed in step S1110 in FIG. 12.

In step S1500, a decision is made as to whether or not the full pressswitch (SW2) 192 is is an ON state. The operation proceeds to step S1510if the full press switch SW2 is determined to be in an ON state, whereasthe operation proceeds to step S1520 if the full press switch SW2 isdetermined to be in an OFF state.

In step S1510, a decision is made as to whether or not exposure startprocessing has been completed. If the exposure start processing isjudged to have been completed, the operation proceeds to step S1520,whereas if the exposure start processing is judged to be incomplete, theoperation proceeds to step S1530. The full press switch SW2 is operatedto trigger the exposure processing. If the exposure has not been startedyet when the switch enters an ON state, the exposure operation starts atthis point and if the exposure has already started, the exposure controlis implemented.

In step S1520, a decision is made as to whether or not exposure is inprogress. The operation proceeds to step S1540 if it is decided thatexposure is in progress, whereas the operation proceeds to step S1570 ifit is decided that exposure is not in progress. In step S1530, theexposure start processing such as raising a mirror (not shown) andopening the shutter is executed. In step S1540, a decision is made as towhether or not the VRSW 194 is in an ON state. The operation proceeds tostep S1550 if the VRSW 194 is determined to be in an ON state, whereasthe operation proceeds to step S1560 if the VRSW 194 is determined to bein an OFF state.

In step S1550, the reference value is integrated. This operation isequivalent to computing the image blur that has not been correctedthrough the optical blur correcting operation. The integrated value isstored into memory or the like to be used when computing the point-imagefunction after the exposure.

In step S1560, the target drive position signal is read and stored intomemory or the like. After the exposure ends, the point-image function iscomputed by using the target drive position signal thus stored. In stepS1570, a decision is made as to whether or not processing for ending theexposure has been completed. The operation proceeds to step S1590 if theprocessing is determined to have been completed, whereas the operationproceeds to step S1580 if the processing is determined to be incomplete.In step S1580, the exposure end processing such as lowering the mirrorand closing the shutter is executed.

In step S1590, a decision is made as to whether or not the computationof the point-image function has been completed. The operation proceedsto step S1610 if the point-image function computation is determined tohave been completed, whereas the operation proceeds to step S1600 if thepoint-image function computation has not been completed.

In step S1600, the computation of the point-image function is started oris continuously executed. If the point-image function computation hasnot been started when the operation proceeds to this step, thecomputation is started, whereas if the computation has already beenstarted, the computation is continuously executed.

In step S1610, a decision is made as to whether or not the imagerestoration computation has been completed. If the image restorationcomputation has been completed, the operation proceeds to step S1120,whereas if the image restoration computation has not been completed, theoperation proceeds to step S1620. In step S1620, the image restorationcomputation is started or is continuously executed. If the imagerestoration computation has not been started when the operation proceedsto this step, the computation is started, whereas if the imagerestoration computation has already been started, the computation iscontinuously executed.

In addition, if the optical blur correcting operation is not executed,the extent of blurring can be reduced by computing the point-imagefunction based upon the output from the angular velocity sensor 10 andexecuting image restoration processing after the photographingoperation. Thus, even when the blur correction lens 70 cannot be engagedin operation due to insufficient battery power, failure to turn on theswitch or the like, the blur can be reduced through the imagerestoration processing.

As explained in detail above, the present invention achieves theadvantages summarized below.

There are provided the blur correction optical system that corrects animage blur, the point spread function computing unit that computes apoint spread function by using a reference value or a vibrationdetection signal and the image restoration computing unit that correctsan image blur through image restoration by executing image processing onan image captured by the image-capturing unit using the point spreadfunction, and it is possible to restore the image regardless of whetheror not an optical blur correction is executed with the blur correctionoptical system.

Since the point spread function computation switching unit is providedthat selects either the reference value or the vibration detectionsignal to be used in the computation of the point spread functionexecuted at the point spread function computing unit, the optimal pointspread function computing method satisfying specific needs can beselected and the image can be restored effectively whenever necessary.

Since the point spread function computation switching unit alsofunctions as a blur correcting operation setting unit that switcheson/off the blur correcting operation by the blur correction opticalsystem, the point spread function computing method can be adjusted byinterlocking with the blur correcting operation by the blur correctionoptical system and the point spread function can be computed in theoptimal manner based upon whether or not the blur correcting operationis executed by the blur correction optical system.

When the blur correcting operation by the blur correction optical systemis not executed, the point spread function computing unit computes thepoint spread function by using the vibration detection signal. The pointspread function thus computed directly relates to the vibration of thecamera and the image blur caused by the vibration of the camera can becorrected through image restoration.

Since the blur correction optical system which corrects an image blurand the point spread function computing unit, which computes the pointspread function by using the reference value or the vibration detectionsignal are provided, the point spread function needed in the imagerestoration can be computed through the optimal computing methodregardless of whether or not the optical blur correction is executedwith the blur correction optical system, and the photographed image canbe subsequently restored with a high level of effectiveness by utilizingan external device or the like.

(Third Embodiment)

A third embodiment includes an image restoring device and a camera bodyprovided independently of each other. The same reference numerals areassigned to components having similar functions to those in the firstembodiment to minimize the need for a repeated explanation thereof.

The following is a detailed explanation of the embodiment of the presentinvention, given in reference to drawings and the like. FIG. 14 is ablock diagram of the system configuration adopted in the thirdembodiment of the blur correction camera according to the presentinvention.

The blur correction camera body 1 in the embodiment adopts a structureachieved by adding a function correcting unit 105, an interface unit 130and an image restoration decision-making unit 140 in the structure inthe first embodiment, with an image restoration computing unit 210constituting part of an image reproduction device 2, which is a separatedevice independent of the camera body 1.

The image reproduction device 2 is an image restoring device connectedwith the image recording unit 120 or the blur correction camera 1 via atransfer cable or the like, which is capable of reproducing andrestoring an image having been captured in the blur correction camera 1.

The function correcting unit 105 corrects the point-image function byusing a position signal indicating the position of the blur correctionlens 70, which is provided by the position detection unit 60. Morespecifically, it corrects the point-image function by using the positionsignal, based upon a control position error representing the differencebetween a target drive position and the position signal (the actualdrive position).

With the interface unit 130 and the image reproduction device 2connected with each other via a transfer cable 300, an image saved atthe image recording unit 120 and information needed for the imagerestoration processing are transferred to the image reproduction device2 as necessary.

The interface unit 130 is a communication means having a terminal towhich the transfer cable 300 used to connect the blur correction camera1 and the image reproduction device 2 is connected.

The connecting cable 300 is used to connect a connecting connector atthe interface unit 130 to a communication port (e.g., an RS-232C, a USB,a parallel port or an IEEEE 1394) at the image reproduction device 2.The blur correction camera 1 and the image reproduction device 2exchange data via this connecting cable 300.

The image restoration decision-making unit 140 executes decision-makingprocessing to decide whether or not to execute a point-image functioncomputation. Namely, it makes a decision based upon informationindicating the shutter speed, the extent of vibration and the like as towhether or not to execute the point-image function computation by usingthe output data provided by the reference value computing unit of theblur correction control unit 30.

Since the image restoration decision-making unit 140 determines whetheror not the point-image function computation needs to be executed, onlythe essential data need to be saved at the image recording unit 122.Thus, redundant arithmetic operations are eliminated and the requiredmemory capacity is reduced.

An operation unit 190 includes a halfway press switch (SW) 191, a fullpress switch (SW) 192 and a blur correction mode selector switch (SW)194.

The blur correction mode selector switch 194 is an operating memberoperated to select an optical correcting operation mode and an imagerestoration mode in a specific combination. The blur correction modeselector switch in the embodiment allows one of three blur correctingoperation mode combinations to be selected through the operationdescribed below.

If a “blur correction off mode” is selected, neither the opticalcorrection or the image restoration is executed. Namely, the drive ofthe blur correction lens 70 is stopped and no blur correcting operationis executed. In addition, no image restoration data are recorded orsaved.

If an “optical correcting operation mode” is selected, only the opticalcorrecting operation is executed by driving the blur correction lens 70and engaging it in an image blur correcting operation. No point-imagefunction for image restoration processing is computed and no imagerestoration data are recorded or saved.

If an “image restoring operation mode” is selected, the opticalcorrecting operation and an operation necessary for the imagerestoration are executed. The image restoration data needed in the imagerestoration processing are transmitted from the optical correctionsystem 500 to the point-image function computing unit 100 via the imagerestoration decision-making unit 140.

Next, the image reproducing device 2 is described.

The image reproduction device 2 includes the image restoration computingunit 210 that executes the image restoration processing, an imagedisplay unit 220 at which an image is displayed and a restoration resultsaving unit 230.

The image reproduction device 2 is achieved by using a personal computerin the embodiment, and the personal computer having installed therein anapplication software program containing a dedicated blur correctionprogram needed to execute the image restoration is enabled to functionas the image reproduction device 2.

The blur correction program includes a data input unit (data input step)that receives image data, a point spread function and various types ofparameters transferred from the camera side and a setting unit thatallows the image reproduction device to set parameters for the camera aswell as an image restoration computing unit (image restorationcomputation step) that executes the image restoration processing.

It is to be noted that instead of using a personal computer, the imagereproduction device 2 may be constituted as a dedicated reproducingdevice or it may be built into the camera.

The image restoration computing unit 210 executes the image restorationprocessing for correcting a blur contained in an image based upon theimage data transmitted from the image recording unit 120 of the blurcorrection camera 1, the point-image function information correspondingto the image data and the various parameters used during the imagerestoration processing.

While the Wiener filter having been described in reference to expression(6) is used in the image restoration processing executed by the imagerestoration computing unit 210, the present invention is not limited tothis example and the image restoration processing may be executed byadopting another method.

The image display unit 220 at which an image photographed by thephotographer or an image resulting from the image restoration isdisplayed, is constituted with the monitor unit of the personal computerin the embodiment.

At the restoration result saving unit 230, the restored image resultingfrom the image restoration processing executed by the image restorationcomputing unit 210 and the parameters used in the image restorationprocessing are saved in correspondence to the raw image.

Next, the basic operation of the blur correction camera 1 executed inthe embodiment is explained;

FIG. 15 presents a flowchart of the basic operation of the cameraexecuted in relation to blur correction.

As the halfway press switch 191 is turned on in step S210, the operationproceeds to step S220. In step S220, a decision is made with regard tothe state of the blur correction mode selector switch 194. The operationproceeds to step S230 to start an optical correcting operation flow ifthe “optical correcting operation mode” has been selected, whereas theoperation proceeds to step S240 to start an image restoration processingflow during which the optical blur correcting operation and the imagerestoration processing operation are executed in combination if the“image restoring operation mode” has been selected. If the “blurcorrection off mode” has been selected, the operation proceeds to stepS250.

In step S250 to which the operation proceeds when the “blur correctionoff mode” has been selected, neither the optical correction nor theimage restoration is executed, the drive of the blur correction lens 70is stopped, no blur correcting operation is executed and no imagerestoration data are recorded or saved.

The following is an explanation of the operations executed in the blurcorrection camera 1 in the “optical correcting operation mode” and inthe “image restoring operation mode”.

First, the operation of the blur correction camera in the “opticalcorrecting operation mode” is explained.

FIG. 16 presents a flowchart of the basic operation of the cameraexecuted in the optical blur correcting operation mode.

In step S400, the cutoff frequency fc at the LPF unit constituting thereference value computing unit 31 is set to 0.1 Hz. In step S410, theangular velocity sensor 10 constituting the vibration detection unit isturned on. In step S420, the blur correction lens 70 having been held inthe lock is released. In step S430, the blur correcting operationstarts. The blur correction started in step S430 is the optical blurcorrecting operation through which the blur is corrected by moving theblur correction lens 70 along a direction substantially perpendicular tothe optical axis so as to cancel out the image blur based upon theoutput from the angular velocity sensor 10.

In step S440, the state of the halfway press timer is detected, and theoperation proceeds to step S450 if the halfway press timer is determinedto be in an OFF state, whereas the operation proceeds to step S470 ifthe halfway press timer is determined to be in an ON state. In stepS450, the blur correcting operation is stopped, and in step S460, thecorrection lens 70 is locked and the operation exits the opticalcorrection mode. In step S470, the state of the full press switch 192 isdetected and the operation proceeds to step S480 if the full pressswitch 192 is in an ON state whereas the operation returns to step S440if the full press switch 192 is in an OFF state.

In step S480, after executing a centering operation for the blurcorrection lens 70, the blur correction is started again. When the blurcorrection lens is not driven by the optical drive unit 50, the opticalaxis of the blur correction lens 70 and the optical axis I of thephotographic optical system are not always aligned. Under normalcircumstances, the blur correction lens 70 will have moved to an end ofits moving range, and if the blur correcting operation is started inthis state, there is bound to be a direction along which the blurcorrection lens cannot be driven. Accordingly, the centering operationis executed to drive the blur correction lens 70 so as to substantiallyalign the optical axis of the blur correction lens 70 with the opticalaxis I of the photographic optical system.

In step S490, the shutter is opened and exposure at the image-capturingunit 110 is started. In step S500, a decision is made as to whether ornot flash light (SB) is to be emitted, and if it is decided that flashlight is to be emitted, the operation proceeds to step S510, whereas ifit is decided that flash light is not to be emitted, the operationproceeds to step S520. In step S510, flash light is emitted. In stepS520, the shutter is closed, thereby ending the exposure. Subsequently,the operation returns to step S440 to execute the halfway press timerdecision-making routine.

Next, the operation of the blur correction camera set in the “imagerestoring operation mode” is explained.

FIG. 17 presents a flowchart of the basic operation of the cameraexecuted in the image restoring operation mode.

In step S600, the cutoff frequency fc at the LPF unit constituting thereference value computing unit 31 is set to 1 Hz. While the cutofffrequency fc is set to 0.1 Hz in the “optical correcting operation mode”as described earlier, a higher cutoff frequency fc is set in the “imagerestoring operation mode”.

By setting the cutoff frequency to a higher level, a greater componentof the vibration to which the blur correction camera 1 is subjected isallowed to evince in the point-image function computed by thepoint-image function computing unit 100 so as to reduce the component toundergo the blur correction executed by driving the blur correction lens70. Since this reduces the extent to which the blur correction lens 70needs to be driven, the blur correction lens 70 can be driven within itsdrivable range with a comfortable margin. While the blur is correctedthrough the optical blur correcting operation to a lesser extent and theblur manifests to a greater extent in the captured image in this case,the blur manifesting to a greater extent in the captured image can becorrected later through the image restoration. Thus, an image with noimage blur or hardly any image blur can ultimately be achieved throughhighly effective blur correction.

By setting the cutoff frequency used in the reference value computationto a higher value in the “image restoring operation mode” compared tothe cutoff frequency set for the “optical correcting operation mode” asdescribed above so as to split the component to undergo the blurcorrection to a component to be corrected through the optical blurcorrection and a component to be corrected through the imagerestoration, even a blur caused by a very shaky hand movement can becorrected in a more effective manner in the “image restoring operationmode” than in the “optical correcting operation mode”.

Since the operational flow from step S610 through step S670 in FIG. 17is similar to the operational flow from step S410 through step S470 inFIG. 16, a detailed explanation is not provided.

In step S680, a decision is made with regard to the restorationprocessing. The restoration processing decision-making in step S680 isto be explained in further detail later in reference to FIG. 18. If theimage restoration processing is judged to be unnecessary through therestoration processing decision-making in this step, the operationproceeds to step S690, whereas the operation proceeds to step S720 ifthe image restoration processing is judged to be necessary.

In step S690, the blur correction resumes after executing a centeringoperation for the blur correction lens 70 as in step S480 in FIG. 16. Instep S700, the shutter is opened and the exposure at the image-capturingunit 110 starts. In step S710, the shutter is closed and the exposureends. Then, the operation returns to step S640 to execute the halfwaypress timer decision-making routine. In step S720, the blur correctionresumes after executing the centering operation for the blur correctionlens 70 as in step S480 in FIG. 16.

In step S730, the shutter is opened and the exposure at theimage-capturing unit 110 starts. In step S740, data to be used in thepoint-image function computation are obtained while the exposure is inprogress. The data to be used in the point-image function computationinclude the reference value computed based upon the output from theangular velocity sensor 10 and error information indicating the errorcomputed based upon the position information indicating the position ofthe blur correction lens 70 and is provided by the position detectionunit 60. The acquisition of the point-image function computation data instep S740 is to be explained in detail later in reference to FIG. 19.

In step S745, the shutter is closed and the exposure ends. In step S750,the point-image function is computed by using the point-image functioncomputation data having been obtained. Since a method that may beadopted when computing the point-image function has been explained inreference to the first embodiment, a repeated explanation is omitted. Instep S755, the point-image function having been computed in step S740 iscorrected by using the error data (operation executed by the functioncorrecting unit 105).

Now, a method that may be adopted when correcting the point-imagefunction by using the error data is explained.

With lc(t) and lr(t) respectively representing the target drive positionand the actual drive position of the blur correction lens 70, each takenas a function of time t, control position errors e(t) manifesting alongthe X axis and the Y axis are given as in the following expressions(operation executed by the control position error output unit).ex(t)=lcx(t)−lrx(t)  (12)ey(t)=lcy(t)−lry(t)  (13)

With e (x, y) representing a function obtained by expanding expressions(12) and (13) two-dimensionally, a corrected point-image function p′ (x,y) is given as in the following expression by using the point-imagefunction p(x, y) presented in expression (1).p′(x, y)=p(x, y)+e(x, y)  (14)

After correcting the point-image function, a blur mark is attached tothe image restoration processing target image in step S760.

In step S770, the corrected point-image function is recorded as blurinformation and then the operation returns to step S640.

Next, the processing of the blur information output from the opticalcorrection system 500 and the acquisition of the point-image functioncomputation data are explained.

FIG. 18 presents a detailed flowchart of the operation executed by theimage restoration decision-making unit 140 (in step S680 in FIG. 17) tomake a decision based upon the blur detection data as to whether or notthe point-image function computation is to be executed.

Based upon the decision made by the image restoration decision-makingunit 140, a decision is made as to whether or not the vibrationdetection data needed for the image restoration are to be recorded.

In step S310, the effectiveness of the image restoration processing isjudged based upon the extent of the vibration having been detected. Inthis step, a decision is made as to whether or not the results of thetarget drive position computation indicate that the blur can beeffectively corrected through the image restoration processing bychecking the image restorable condition range set in-advance based uponthe vibration information and camera photographing information.

For instance, if the blur manifests to an excessive extent (blurmanifesting to the highest limit extent), noticeable stripes will stillremain in the image after the image restoration processing and thus, theimage quality will still be poor. If, on the other hand, the extent ofblurring is insignificant (blur manifesting to the lowest limit extent),the image quality will not be significantly improved through the imagerestoration.

Accordingly, these blurring extent limits are set in advance byconducting tests or based upon experience.

In step S320, a decision is made as to whether or not the imagerestoration processing needs to be executed based upon the shutter speed(the length of the exposure period). Since the extent of blurring can bepredicted with some accuracy based upon the shutter speed, a decision ismade as to whether or not the image restoration processing needs to beexecuted based upon the predicted blurring extent in this step. When theshutter speed is high, only a very small extent of blurring, if any,occurs, and accordingly, the quality of the image is judged to beacceptable. This blurring extent can be determined in correspondence toboth the focal length and the shutter speed. It is generally acceptedthat when the optical blur correction is not executed, a blurattributable to an unsteady hand movement manifests when the shutterspeed is lower than (1/focal length). However, since the optical blurcorrection is also executed in the embodiment, the image restorationprocessing should be executed only if the relationship expressed in (15)below, for instance, is true.(A/focal length)<shutter speed (exposure period)  (15)

“A” in expression (15) may be a specific value or it may be a variablethat changes in correspondence to other conditions.

If it is decided through the shutter speed decision-making and thedetected blurring extent decision-making in steps S310 and S320 that therestoration processing needs to be executed, the operation proceeds tostep S330 to enter the restoration processing in step S720 in FIG. 17.

If, on the other hand, it is decided through the shutter speeddecision-making or the detected blurring extent decision-making in stepS310 or step S320 that the restoration processing is not necessary, theoperation proceeds to step S340 to issue a warning or bring up a display(message) indicating that the image restoring operation is not to beexecuted. The message may be provided as, for instance, a warning soundor through a specific display.

After executing step S340, the operation proceeds to step S350 to enterthe non-restoration processing exposure sequence in step S690 in FIG.17.

By making a decision as to whether or not the image restoration shouldbe executed as shown in FIG. 18, the required memory capacity can bereduced since the memory only needs to store a smaller volume of blurinformation for the image restoration processing.

FIG. 19 presents a detailed flowchart of the operation executed toobtain the point-image function computation data (in step S740 in FIG.17).

In the embodiment, the reducing processing (processing executed at aninformation volume reducing unit) shown in FIG. 19 is executed primarilyin order to save space in the memory.

Following the exposure start, counters are reset in step S910. Morespecifically, a counter N is set to land a counter K is set to 0. Thecounter N indicates the value assigned to one of a plurality ofreference values to distinguish it from the rest of the referencevalues, whereas the counter K functions as a timer that measures time.

In step S920, a first reference value output, i.e., ω0(1), is saved.

In step S925, the corresponding error e(1) is computed and saved. The“error” as referred to in this context represents the difference(control position error) between the target drive position for the blurcorrection lens 70 computed by the target drive position computing unit33 and the actual drive position of the blur correction lens 70 outputby the position detection unit 60, which is computed by the blurcorrection control unit 30. While the blur correction control unit 30outputs a drive signal that will make-up for the difference between atarget drive position and the actual drive position, the blur correctionlens 70 cannot always achieve the drive target and an error occurs undersuch circumstances.

In step S930, the reference value output average ω0ave is computed asexpressed below.ω0ave={ω0(N)+ω0ave×(N−1)}/N  (16)

In step S940, one of the counters is checked. If the counter K indicates100, the operation proceeds to step S950, but the operation otherwiseproceeds to step S970. In step S950, the reference value output ω0(N) issaved.

In step S955, the corresponding error e(N) is computed and saved. Instep S960, the timer counter K is cleared to zero. In the embodiment,the sampling frequency at the angular velocity sensor 10 is 1 kHz andthe reference value output is saved every 0.1 sec, thereby culling thereference value outputs. In step S970, it is checked to determine as towhether or not the shutter is closed, and the operation proceeds to stepS990 if the shutter is open, whereas the operation proceeds to step S980if the shutter is closed.

In step S980, the last reference value output ω0(N) is saved. The mostrecent reference value output is saved so as to ensure that at leastanother reference value output is saved in addition to the initialreference value output even when the reference value outputs are culledwith the shutter set at a high speed setting. For instance, since thereference value output is saved every 0.1 sec when the samplingfrequency is set to 1 kHz in the embodiment, only the first referencevalue output will have been saved if the shutter speed is set higherthan 1/10 sec, and in such a case, a point-image function cannot becomputed. In addition, the reference value output average ω0ave havingbeen computed in step S930 is also saved in step S980.

In step S985, the corresponding error e(N) is computed and saved. Insteps S990 and S1000, the counters are incremented before the operationreturns to step S930 to compute the reference value output average.

Now, the reducing processing mentioned earlier is explained.

In the embodiment, the point-image function used in the imagerestoration processing is computed based upon the reference valueoutput. Since the reference value output is an LPF output with a 1 Hzcutoff frequency (in the image restoration flow in FIG. 17) as describedearlier, the frequency level is lower than the level of the frequencycomponent of the blur caused by the unsteady hand movement. For thisreason, the sets of data to be used in the point-image functioncomputation can be reduced.

If the point-image function were computed by using all the blurdetection data provided by the optical correction system 500, a hugeload of computation must be executed and a huge memory capacity isrequired.

For instance, the number of sets of blur detection data obtained as theresults of the target position computation when the sampling frequencyfor the reference value computation is 1 kHz amounts to N=1000 sets ofreference value data per second, which represents an extremely largevolume of data. The frequency of an unsteady hand movement is normallywithin the range of 0.1 to 10 Hz, and the cutoff frequency of the lowpass filter installed in the reference value computing unit 31 thatcalculates the reference value for the vibration caused by an unsteadyhand movement is approximately 1 Hz. Namely, the frequency componentequal to or lower than 1 Hz constitutes the primary component processedat the point-image function computing unit 100. The frequency of 1 Hzcan be adequately expressed by using data taken with a frequencyapproximately 10 times as high, i.e., in the 0.1 sec cycle. Thus, it ispossible to reduce the data sampled with the 1 kHz frequency by thefactor of 1/100.

In addition, when the cutoff frequency at the LPF used in the referencevalue output computation is altered, the reducing extent, too, needs tobe adjusted in correspondence to the altered cutoff frequency.

Through the processing described above, the length of time required forthe computation processing, the required memory capacity and the likecan be reduced.

Following the reducing processing, the blur information is recorded intothe recording medium to be used in the image restoration processingexecuted by the image reproducing device or the data are transferred tothe image reproducing device 2. Since only the absolute minimum numberof sets of data required in the image restoration processing, resultingfrom the reducing processing, is recorded or transferred in theembodiment, great advantages such as reductions in the length of timerequired for the data transfer and in the length of time required forthe computation processing and in particular, more efficient use ofmemory capacity are achieved.

Now, the operation of the point-image function computing unit 100executed in step S750 in FIG. 17 is explained.

The problem that some extent of residual blurring still manifests in theimage (occurrence of a blur correction residual error) even after theblur correction by the optical correction system 500 has been describedin reference to the related art. Such a blur correction residual erroris primarily attributable to the reference value and the error occurringbetween the actual drive position of the blur correction lens and thetarget drive position. Accordingly, the point-image function computingunit 100 in the embodiment calculates a point-image function based uponthe reference value, which is then corrected based upon the error by thefunction correcting unit 105. The point-image function thus corrected istransmitted to the image restoration computing unit 210. The imagerestoration computing unit 210 executes an image restoration computationbased upon the corrected point-image function transmitted thereto so asto correct the image blur that the blur correcting operation by the blurcorrection lens 70 has failed to correct and, as a result, a highquality image in which blur has been effectively corrected is obtained.

Next, the operation of the image reproducing device 2 is explained.

FIG. 20 presents a flowchart of the basic operation of the imagereproducing device 2.

It is assumed that the blur correction program in conformance to whichthe image restoration is executed is preinstalled in the imagereproducing device 2.

As explained earlier, the image data generated on the camera side aretransferred to the image reproducing device 2 via the transfer cable 300in the embodiment.

It is assumed that before the processing in FIG. 20 starts, images havealready been transferred, the blur correction (image restorationprocessing) program has been started up and a menu screen has beenbrought up on display.

In step S2010, a restoration processing button is clicked with an inputdevice such as a mouse and in response, the operation enters the imagerestoration flow. Since the blur mark was attached and recorded inadvance at the camera to each image determined to be a target image toundergo the restoration processing, only images appended with the blurmark are read out and displayed as the operation for reading images tobe reproduced starts in step S2020. In step S2030, the user selects anddisplays an image to undergo the image restoration processing bychecking the images or the various parameters related to their imageblurs.

In step S2040, blur locus data and the image point blur constituting theparameters needed for the image restoration of the selected image aredisplayed in further detail. More specifically, correction informationhaving been recorded by the blur correction camera 1, such as the blurlocus data and the image point blur, the photographing information andthe like are displayed on the image display unit (display) 220 so as toenable the operator to work directly with the blur locus data on theimage display unit 220 as desired.

FIGS. 21 and 22(a) to 22(c) represent a specific example of how imagesmay be displayed and the various parameters may be adjusted. FIG. 22(a)shows the blur locus data, FIG. 22(b) shows the blur locus data in arough adjustment operation mode and FIG. 22 (c) shows the blur locusdata in a fine adjustment operation mode.

In step S2050, the parameters used in the image restoration are adjustedand set as desired. In step S2060, the restoration processing isexecuted by using the parameters having been set in step S2050. In stepS2070, the pre-restoration blurred image and the restored imageresulting from the restoration processing are displayed for comparisonon the image display unit 220 at the image reproduction device 2. Instep S2080, the pre-restoration blurred image and the restored imageresulting from the restoration processing are visually compared and adecision is made as to whether or not the restored image is acceptable(whether or not the image restoration should be re-executed). Theoperation proceeds to step S2085 if the restored image is acceptable,whereas the operation returns to step S2040 if the image restoration isto be re-executed.

In step S2085, the user makes a decision as to whether or not therestored image and the parameters should be saved. The operationproceeds to step S2090 if the restored image and the parameters are tobe saved, whereas the processing ends if they are not to be saved. Instep S2090, the user makes a decision as to whether or not to save therestored image and parameters by writing the data over other data and aninstruction is issued accordingly. The operation proceeds to step S2110if the data are not to be saved by writing them over other data, whereasthe operation proceeds to step S2100 if they are to be saved through anoverwrite. In addition, if the data are to be saved through anoverwrite, the data to be written over and thus deleted (existing saveddata) are also selected.

In step S2100, the past restored image and the corresponding parameters(the data having been selected in step S2090 to be written over) havingbeen saved in correspondence to the raw image are deleted. In stepS2110, the current restored image and the new parameters used for thecurrent image restoration processing are saved in correspondence to theraw image.

In the step, three different files, a first file containing thepre-restoration raw image (blurred image), a second file containing theparameters and a third file containing the restored image are preparedand information related to the file containing the raw image and thefile containing the restored image is written in the file containing theparameters. As the file containing the parameters is subsequently openedat the image reproduction device 2, the file containing the raw imageand the file containing the restored image, which are correlated to thefile containing the parameters, too, are opened and displayed.

FIG. 23 schematically illustrates the relationship among restoredimages, parameters and the raw image saved in step S2110.

As shown in FIG. 23, a plurality of combinations of restored images andparameters all correlated to a single raw image exist in the embodiment.

Since this allows the parameters to be saved in correspondence to eachimage no matter how many times image restoration processing is executed,the user is able to work with his images smoothly without becomingconfused.

It is to be noted that the restored image, the parameters and the rawimage can be saved by adopting any of various other modes instead of themode adopted in the embodiment. Table 1 below presents examples ofpossible modes that may be adopted, each corresponding to a specificcombination of instructions for saving/not saving the images and theparameters. TABLE 1 SAVE SAVE RAW RESTORED SAVE SAVING IMAGE? IMAGE?PARAMETERS? METHOD No. NOT SAVED SAVED NEW 1 SAVED OVERWRITE 2 NOT SAVEDSAVED NEW 3 OVERWRITE 4 SAVED NOT SAVED NEW 5 OVERWRITE 6 SAVED SAVEDSAVED NEW 7 OVERWRITE 8 NOT SAVED SAVED NEW 9 OVERWRITE 10 SAVED NOTSAVED NEW 11 OVERWRITE 12

In modes No. 1 through 6 in Table 1, the raw image is not saved.

For instance, if the user is satisfied with the restoration results anddoes not wish to have the image restoration re-executed, there may notbe any need to save the raw image. Mode No. 1, 2, 5 and 6 will beeffective under such circumstances.

If, on the other hand, the user is not satisfied with the restored imageobtained by executing the restoration processing on a raw image and doesnot wish to save either the raw image or the restored image, the usermay still wish to save the parameters to be used for reference whenanother raw image subsequently undergoes the restoration processing.Mode No. 3 and 4 will be effect of in such a situation.

In mode No. 7 through 12 in Table 1, the raw image is saved. Mode No. 7and 8 among them are adopted in the embodiment described above.

For instance, if the processing speed at the image restoration computingunit 210 of the image reproduction device 2 is high, only the parametersmay be saved in correspondence to a raw image so that the imagerestoration can be executed by using the parameters and the raw imagehaving been saved to display the restored image whenever necessary,without saving the restored image. As a result, memory capacity at therecording medium or the like can be saved while assuring ease of usecomparable to that achieved in the embodiment (modes 9 and 10).

In addition, even when the user is satisfied with the restorationresults and does not think it necessary to re-execute the imagerestoration under the same conditions, the user may still wish to savethe raw image just to ensure that if such needs ever arise, the imagerestoration can be re-executed by using different parameters. Modes 11and 12 are effective under such circumstances.

It is to be noted that in each of the combinations described above, theuser is allowed to choose whether to save the data as additional data orthrough an overwrite.

In the example presented in FIG. 21, a window display that includes thepre-restoration blurred image, the restored image obtained through therestoration processing, information related to the point-image functionand the blur locus data corresponding with one another is provided onthe image display unit 220. By displaying them in a single screen tofacilitate comparison, the operator is able to intuitively determinewhich image area needs to be corrected instantly.

In addition, the lower right display in FIG. 21 enables the operator tomanipulate the blur locus data. The blur locus data displayed on theimage display unit 220 in this manner in the embodiment can be locallymanipulated by the operator with a mouse or the like. The restorationprocessing is re-executed based upon the blur locus data having beenmanipulated as described above so as to make a decision with regard tothe finer details through comparison.

In the embodiment, the blur locus data having been obtained, which isshown in FIG. 22(a), can be reduced/enlarged in reference to a point P0indicated with the mouse so as to allow the image data to be processedthrough a rough adjustment as shown in FIG. 22(b) or through a fineadjustment as shown in FIG. 22(c). FIG. 22(c) shows an example of animage data operation in the fine adjustment mode in which data can bemanipulated by using a larger number of sets of data to facilitate theevaluation of the parameters of the resulting restored image. This, inturn, improves the level of freedom in the image operation and improvesthe efficiency of the processing.

In the point-image function computation in the related art, the outputfrom a sensor such as the angular velocity sensor 10 is directly used inthe computation, and for this reason, the point-image function containsnumerous error factors, making it difficult to obtain a high-qualityimage even by manipulating the image on display. In contrast, the imagerestoration processing in the embodiment is executed through apoint-image function computation in which output data with a lesserextent of noise error, resulting from the blur correction achievedthrough the optical blur correcting operation are used and thus, anextremely high quality restored image is achieved. In addition, sincethe blur locus data, the image point data and the like of the image canbe directly manipulated by using, for instance, a mouse, the effect ofthe parameters used in the image restoration processing on the imagerestoration results can be evaluated with greater ease and highlyefficient processing can be executed.

As described above, the blur information is recorded in correspondenceto the image in the embodiment. As a result, the user is able to verifythe blur information simply by viewing the image at the imagereproduction device 2 (image viewing software program). Accordingly, theuser does not need to correlate the image with the blur informationprior to the image restoration, and the work efficiency is improved. Thework efficiency is further improved with the display of the blur markindicating that image restoration is required.

In the embodiment, the actual drive position of the blur correction lens70 is detected, the difference between the actual drive position and thetarget drive position is determined as an error and a point-imagefunction reflecting the error is computed. Then, by restoring the imageusing the point-image function, the blur correction residual errorattributable to the drive error occurring while driving the blurcorrection lens 70, too, can be corrected through the image restoration,thereby enhancing the blur correction effect.

It is to be noted that the present invention is not limited to theembodiment described above and allows for numerous variations andmodifications which are equally considered to be within the scope of thepresent invention.

For instance, while a digital low pass filter is used in the referencevalue computation in the embodiment, the present invention is notlimited to this example and the reference value computation may beexecuted by adopting another method such as the moving average method.While the cutoff frequency at the LPF is adjusted depending upon whetheror not the image restoration is to be executed, the present invention isnot limited to this example and the cutoff frequency at the LPF mayremain unchanged regardless of whether or not the image restoration isexecuted.

While the blur correction camera 1 and the image reproduction device 2are connected with each other via the transfer cable 300 to enable dataexchange in the embodiment, the present invention is not limited to thisexample and instead, a multipurpose recording medium having recordedtherein an image having been photographed with the blur correctioncamera 1, the point-image function corresponding to the image, the otherparameters needed in the image restoration processing and thephotographing information may be used instead, or the data may betransmitted through wireless communication.

While the restoration result operation unit 230 is included in the imagereproduction device 2 in the embodiment, the present invention is notlimited to this example and the restoration result operation unit may beinstalled on the camera side as long as the camera includes an imagerestoration computing unit.

While the data reducing processing is executed prior to the point-imagefunction computation in the embodiment, the present invention is notlimited to this example and the data reducing processing may be executedafter the point-image function computation.

The blur correcting operation mode selector switch 194 is athree-position switch in the embodiment. However, the present inventionis not limited to this example and it may be an ON/OFF switch operatedto individually turn on/off the “optical correcting operation mode” andthe “image restoring operation mode” or it may be a switch in software.If the “image restoring operation mode” alone is selected withoutselecting the “optical correcting operation mode”, as well through sucha switch, a warning for the user may be issued by generating a warningsound, bringing up a warning display or appending a warning mark to theimage to indicate that the image blur has not been optically corrected.

In the embodiment, if the operator having checked the need for imagerestoration determines that the image restoration is not to be executed,the blur information such as the point-image function is neitherobtained through an arithmetic operation nor recorded (saved). Thepresent invention is not limited to this example and the blurinformation having a mark (a warning) attached thereto to indicate thatthe image is not suited for image restoration may be recorded or a markindicating that the blur information has not been recorded/saved may beappended.

In the embodiment, three separate files, i.e., the file containing theraw image, the file containing the parameters and the file containing arestored image are prepared, with the information related to the rawimage file and the restored image file written in the parameter file.However, the present invention is not limited to this example, and thethree types of data may be saved in a single file, for instance, orinformation indicating the correlation among the three types of data maybe saved in another file so as to enable a display of the threedifferent types of data in correlation to one another in reference tothis separate file in an application program.

The advantages that can be obtained in the present invention explainedin detail above are summarized below.

Since the system includes a camera having the point spread functioncomputing unit and an external device having the image restorationcomputing unit, the camera does not need to execute image restorationwhich entails a great load of arithmetic operations, and thus, thecamera can be provided as an inexpensive unit and the level of powerconsumption in the camera, too, is reduced.

Since it includes the point spread function output means for outputtingto the external device the computed point spread function by the pointspread function computing unit via the image recording unit or acommunication means, the image can be restored at the external devicewith ease without requiring a complicated operation.

As the point spread function computing unit computes the point spreadfunction based upon the results of the computation executed at thereference value computing unit, the residual blur which has not beencorrected through the blur correction by the blur correction opticalsystem can be expressed as a point spread function and the blur, whichhas not been corrected through the blur correction by the blurcorrection optical system can be corrected through the imagerestoration.

Since it includes the data input unit that receives the image data andthe point spread function and the image restoration computing unit thatcorrects an image blur through image restoration, an image blur can becorrected after the photographing operation through the imagerestoration executed on the image data containing a blur by using thepoint spread function.

Since the blur correction program includes a data input step in whichimage data and a point spread function are received and an imagerestoration computation step in which an image blur is corrected throughimage restoration, the image restoration can be executed on amultipurpose computer. Thus, the image can be restored without having touse a dedicated external device and the overall system can be achievedat low cost.

Since the system includes the information volume reducing unit thatreduces the volume of information related to the reference value used inthe computation of the point spread function and/or the computed pointspread function, the length of time required for the arithmeticprocessing can be reduced and memory space can be saved. Thus, imagerestoration can be executed without having to employ a high-speedarithmetic processing unit, a large-capacity recording medium, ahigh-speed recording means or a high-speed communication means.

Since the information volume reducing unit reduces the informationvolume by culling the data related to the reference value and/or thecomputed point spread function, the volume of the information is reducedwith ease and reliability.

Since the information volume reducing unit reduces the informationvolume by ensuring that there will be a sufficient volume of informationafter the reducing processing to enable the image restorationcomputation, the quality of the restored image is not compromised and ahigh-quality restored image is obtained.

The system having the restoration result saving unit or the restorationresults saving step in which the parameters used in the image processingexecuted at the image restoration computing unit and/or the restoredimage is saved in correspondence to the raw image, facilitates highlyorganized management of the image data and the parameter settings whenthe user needs to adjust a parameter to be used in image restoration.

Since the restoration result saving unit is capable of saving therein aplurality of sets of parameters each corresponding to one of a pluralityof restored images and/or the restored images themselves, the records ofa plurality of image restoration trials executed by adjusting theparameters can be retained, making it possible to keep track of thehistory of the restoration trials.

Since the blur correction control unit adjusts the details of thecontrol implemented on the blur correction optical system incorrespondence to the mode selection made through the blur correctionmode selection unit, an optimal optical blur correcting operation thatwill work best is executed in conjunction with the image restoringoperation, whenever the image restoring operation mode is selected.

The blur correction control unit adjusts the contents of the controlimplemented on the blur correction optical system by switching themethod adopted to compute the reference value in correspondence to themode selection made through the blur correction mode selection unit. Asa result, when the image restoration is to be executed, the extent towhich the blur is corrected through the optical blur correctingoperation can be adjusted. Even a blur caused by a very shaky handmovement can be corrected through the optimal blur correcting operationby reducing the extent to which the blur is corrected through theoptical blur correcting operation. Since the residual image blur iscorrected through the image restoration, a high-quality image in whichthe blur has been fully corrected can ultimately be obtained.

Since the blur correction control unit adjusts the contents of thecontrol implemented on the blur correction optical system by adjustingthe cutoff frequency at the low pass filter, the extent to which theblur is to be corrected through the optical blur correcting operationcan be altered with ease, and thus, the present invention can beembodied with ease.

When the selection made by the blur correction mode selection unitindicates that the image restoration is to be executed, the blurcorrection control unit sets the cutoff frequency to a higher level thanthe cutoff frequency set when a non-image restoration mode has beenselected at the blur correction mode selection unit. Consequently, theextent to which the blur is to be corrected through the optical blurcorrecting operation is reduced when the image restoration, too, is tobe executed. As a result, even a blur attributable to a very shaky handmovement can be corrected in an optimal manner.

Since the system includes the image restoration decision-making unitthat makes a decision as to whether or not the image restoration modeshould be selected, the image restoration is executed only if the imagequality is likely to be improved through the image restoration and noredundant image restoration is implemented.

The image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe vibration detection signal, and thus, it is capable of judging theextent of the image blur, e.g., an extreme image blur or a very slightimage blur.

Since the image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe shutter speed, the image restoration is executed if the shutterspeed is set to a level at which a concern for an image blur exists.

Since the image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe focal length of the photographic optical system, the imagerestoration is executed if a concern for an image blur exists at theparticular focal length.

Since the image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe point spread function, the need for the image restoration can bejudged with a higher level of accuracy as the extent of blurring changesduring the exposure.

Since the system includes the reporting means for reporting the resultsof a decision made by the image restoration decision-making unitindicating that the image restoration mode should not be entered, thephotographer is able to ascertain the photographing state with ease andthus, the operability is improved.

If the image restoration decision-making unit determines that the imagerestoration mode should not be entered, the image restoration mode isnot entered, thereby ensuring that no redundant arithmetic operation orthe like is executed. This feature is particularly effective inimproving the processing speed and minimizing the power consumption.

If the image restoration decision-making unit determines that the imagerestoration mode should not be entered, the point spread function is notsaved, and thus the required memory capacity can be reduced.

Since the blur correction mode selection unit invariably selects theoptical blur correction mode in conjunction with the image restorationmode, the blur of the image to undergo the image restoration is firstreduced through the optical blur correction to ensure that ahigh-quality image is obtained through the image restoration.

Since the blur correction mode selection unit is not allowed to enterthe image restoration mode without also selecting the optical blurcorrection mode, an erroneous image restoration of a photographic imagethat has not undergone the optical blur correction is prevented.Accordingly, whenever the image restoration is executed, a high-qualityimage results.

A warning is issued if the image restoration mode is selected withoutalso selecting the optical blur correction mode at the blur correctionmode selection unit, an erroneous image restoration of a photographicimage that has not undergone the optical blur correction is prevented.Accordingly, whenever the image restoration is executed, a high-qualityimage results.

The computation of the point spread function by the point spreadfunction computing unit is enabled as the optical blur correction meansis engaged in operation. Thus, the blur in the image to undergo theimage restoration is first reduced through the optical blur correctionand the computed point spread function contains ample information neededin the image restoration to ensure that whenever the image restorationis executed, the resulting image invariably achieves a high imagequality.

(Fourth Embodiment)

The following is an explanation of the fourth embodiment of the presentinvention, given in reference to FIG. 24.

The fourth embodiment differs from the third embodiment in that anoperation equivalent to that executed in step S680 (the imagerestoration decision-making unit) in FIG. 17 in the third embodiment isexecuted after the point-image function computation (step S750), butotherwise, it is identical to the third embodiment. Accordingly, thesame reference numerals are assigned to components having functionssimilar to those in the third embodiment to minimize the need for arepeated explanation thereof.

After executing the point-image function computation in step S750, adecision is made in step S755 as to whether or not the image restorationshould be executed (whether or not the image restoring operation modeshould be selected. While the objective of the operation executed inthis step (the operation of the image restoration decision-making unit)is similar to that of the operation executed in step S680 in FIG. 17 inthe third embodiment, a different decision-making method is adopted (seeFIG. 25). If it is decided in step S755 that the image restoration is tobe executed, the operation proceeds to step S760, whereas if it isdecided that the image restoration is not to be executed, the operationreturns to step S640.

FIG. 25 is a detailed flowchart of the operation executed at the imagerestoration decision-making unit to make a decision based upon thepoint-image function as to whether or not the image restoration is to beexecuted, and the flowchart corresponds to the flowchart presented inFIG. 18 which shows the operation executed in the third embodiment.

In step S5310, a decision is made based upon the point-image function asto whether or not the image restoring operation mode should be selected.If it is decided that the image restoration is to be executed, theoperation proceeds to step S5320 to enter an exposure sequence whichincludes the image restoration (S760 in FIG. 24). If, on the other hand,it is decided that the image restoration is not to be executed, theoperation proceeds to step S5330.

In the specific method that may be adopted when making a decision basedupon the point-image function as to whether or not to select the imagerestoring operation mode, the width of the point-image function may becomputed and a decision that the image restoration is to be executed maybe made if the computed width is smaller than a predetermined value(e.g., 30 μm). The width of the point-image function may be taken as thelength of the diagonal contacting a rectangle obtained by circumscribingthe point-image function.

In step S5330, a warning display (message) indicating that the imagerestoring operation is not to be executed is provided. The message maybe provided, for instance, in the form of a warning sound or a specificdisplay.

In step S5340, the operation enters an exposure sequence that does notinclude restoration processing (S640 in FIG. 24).

In the third embodiment, a decision is made based upon the informationindicating the extent of the detected blur, the shutter speed, the focallength and the like as to whether or not to execute the imagerestoration prior to the point-image function computation. However,since this information is obtained during the period of time elapsingbetween the halfway press operation and the full press operation of theshutter release button, it is used to make a decision by predicting theextent of vibration to occur during the exposure, which means that ifthe extent of vibration blur changes during the period of time elapsingbetween the full press operation of the shutter release button and theexposure end, an accurate decision may not be made. Accordingly, adecision as to whether or not to select the image restoring operationmode is made in the embodiment based upon a point-image functioncomputed by using information obtained during the exposure so as toenable decision-making that reflects the actual vibration informationobtained during the photographing operation.

In addition, since the extent of the image blur manifesting on the imageplane is indicated with a specific numerical value through thepoint-image function, a decision as to whether or not the imagerestoration should be executed can be made with a higher level ofaccuracy.

Since the decision as to whether or not to select the image restoringoperation mode is made based upon the point-image function, the need forthe image restoration can be judged by taking into consideration anychange in the extent of vibration that may take place during theexposure and an even more accurate judgment can be made accordingly.

The advantages that can be obtained through the present inventiondescribed in detail about are summarized below.

Since the system includes the image restoration decision-making unitthat makes a decision as to whether or not the image restoration modeshould be selected, the image restoration is executed only if the imagequality is likely to be improved through the image restoration and noredundant image restoration is implemented.

The image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe vibration detection signal, and thus, it is capable of judging theextent of an extreme blur, e.g., a major image blur or a very slightimage blur.

Since the image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe shutter speed, the image restoration is executed if the shutterspeed is set to a level at which a concern for an image blur exists.

Since the image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe focal length of the photographic optical system, the imagerestoration is executed if a concern for an image blur exists at theparticular focal length.

Since the image restoration decision-making unit makes a decision as towhether or not the image restoration mode should be selected based uponthe point spread function, the need for the image restoration can bejudged with a higher level of accuracy as the extent of vibrationchanges during the exposure.

Since the system includes the reporting means for reporting the resultsof a decision made by the image restoration decision-making unitindicating that the image restoration mode should not be entered, thephotographer is able to ascertain the photographing state with ease andthus, the operability is improved.

If the image restoration decision-making unit determines that the imagerestoration mode should not be entered, the image restoration mode isnot entered, thereby ensuring that no redundant arithmetic operation orthe like is executed. This feature is particularly effective inimproving the processing speed and minimizing the power consumption.

If the image restoration decision-making unit determines that the imagerestoration mode should not be entered, the point spread function is notsaved, and thus the required memory capacity can be reduced.

(Fifth Embodiment)

In the fifth embodiment, the present invention is adopted in a camerathat allows the use of exchangeable photographic lenses. The samereference numerals are assigned to components having functions similarto those in the third embodiment to minimize the need for a reputedexplanation thereof.

FIG. 26 is a block diagram of the system configuration adopted in thefifth embodiment of the blur correction camera according to the presentinvention.

At a camera body 101, a body-side control unit 80A, a power supply unit90, a point-image function computing unit 100, an image-capturing unit110, an image recording unit 120, an interface unit 130, a blurcorrection mode decision-making unit 145, an exposure control unit 150,an electronic flash control unit 180, an operation unit 190 and the likeare disposed.

At an interchangeable lens 102, a lens-side control unit 80B, a RAM 121,a focus lens position detection unit 160, a focal length detection unit170, a blur correction mode selector switch 194, an optical correctionsystem 500 and the like are disposed.

In addition, a signal transfer unit 310 is disposed at a position atwhich the camera body 101 and the interchangeable lens 102 are connectedwith each other so as to enable signal exchange between the camera body101 and the interchangeable lens 102.

An image reproduction device 2 includes a function correcting unit 240.

Next, the operations executed at the camera body 101 and theinterchangeable lens 102 in the embodiment while an image isphotographed are explained.

It is to be noted that while the three modes, the “blur correction offmode” the “optical correcting operation mode” and the “image restoringoperation mode”, can be selected with the blur correction mode selectorswitch 194, as explained in reference to the third embodiment, theexplanation focuses on the operations characterizing the presentinvention, i.e., the operations executed in the “optical correctingoperation mode” and the “image restoring operation mode”.

FIGS. 27 and 28 present a flowchart of the operations executed at thecamera body 101 and the interchangeable lens 102 in the embodiment whilean image is photographed, featuring the blur correction executed inresponse to a normal single-shot shutter release. Since the flowchart ishighly comprehensive, it is presented in two separate parts in FIGS. 27and 28. In addition, in the flowchart, the operational flow (stepsassigned with step numbers in the 3000's) at the camera body 101 isshown on the left side and the operational flow (steps assigned withstep numbers in the 4000's) at the interchangeable lens 102 is shown onthe right side, with a dotted line connecting a pair of steps indicatingthat the operations are executed substantially simultaneously.

In step S3010, the interchangeable lens 102 is allowed to use the powerfrom the power supply unit 90.

Upon being allowed the use of the power (S4010), the interchangeablelens 102 supplies the power to the angular velocity sensor 10 and othercircuits in step S4020.

In step S3020, a decision is made as to whether or not the halfway pressswitch 191 is in an ON state, and the operation proceeds to step S3030if the halfway press switch 191 is determined to be in an ON statewhereas the decision-making in step S3020 is repeatedly executed if thehalfway press switch 191 is determined to be in an OFF state. In stepS3030, a command constituting a start instruction for a blur correctingoperation (hereafter referred to as a halfway press blur correctingoperation) to be executed while the halfway press switch 191 remains inan ON state is transmitted to the lens-side.

Upon receiving the halfway press blur correction start command (S4030),the interchangeable lens 102 releases the lock on the blur correctionlens 70 and starts the halfway press blur correcting operation in stepS4040.

In step S3040, a decision is made as to whether or not the full pressswitch 192 is in an ON state, and the operation proceeds to step S3050if the full press switch 192 is determined to be in an ON state, whereasthe operation returns to step S3030 to repeatedly execute thedecision-making in step S3040 if the full press switch 192 is determinedto be in an OFF state. In step S3050 to which the operation proceedswhen the full press switch 192 is in an ON state, a command constitutinga start instruction for an exposure-in-progress blur correction istransmitted to the lens side. In step S3060, an exposure preparationsuch as a mirror-up operation is executed, the exposure is started instep S3070 and then the exposure ends in step S3080.

Upon receiving the exposure-in-progress blur correction start command(S4050), the interchangeable lens 102 starts an exposure-in-progressblur correcting operation in step S4060. In addition, in step S4060, theposition detection unit 60 at the interchangeable lens 102 detects theactual drive position of the blur correction lens 70 while theexposure-in-progress blur correcting operation is underway, and errorinformation (error data) indicating the difference between the actualdrive position and the target drive position and vibration dataconstituted with information provided by the angular velocity sensor 10are continuously stored into the RAM 121 at least during the period oftime elapsing between the exposure start point and the exposure endpoint.

In step S3090, a command instructing a blur correction stop (a blurcorrection stop command) is transmitted to the interchangeable lens 102.Upon receiving the blur correction stop command (S4070), theinterchangeable lens 102 stops the blur correction control in stepS4080.

In step S3100, post-exposure processing such as a mirror-down operationand a charge is executed. In step S3110, a blur correction lens lockcommand constituting an instruction for locking the blur correction lens70 is transmitted to the interchangeable lens 102. Upon receiving theblur correction lens lock command (S4090), the interchangeable lens 102locks the blur correction lens in step S4100.

In step S3120 in FIG. 28, the blur correction mode decision-making unit145 at the camera body 101 makes a decision as to whether the “opticalcorrecting operation mode” (in which only the optical blur correction isexecuted) or the “image restoring operation mode” (in which both theoptical blur correction and the image restoration are executed) has beenselected with the blur correction mode selector switch 194 disposed atthe interchangeable lens 102. The operation proceeds to step S3130 ifthe “image restoring operation mode” has been selected, whereas theoperation proceeds to step S3170 if the “optical correcting operationmode” has been selected.

In addition, in correspondence to the operation executed in step S3120,the operation proceeds to step S4120 if the “image restoring operationmode” has been selected but the operation proceeds to step S4150otherwise on the interchangeable lens side (S4110).

In step S3130, a command requesting the error data and the vibrationdata having been stored into the RAM 121 earlier is transmitted to theinterchangeable lens 102. Upon receiving the command requesting theerror data and the vibration data (S4120), the interchangeable lens 102transmits the error data to the camera body 101 in step S4130.

In step S4140, a decision is made as to whether or not the transmissionof both types of data has been completed and the operation proceeds tostep S4150 if the data transmission has been completed, whereas theoperation returns to step S4130 if the data transmission has not beencompleted, to continuously transmit the error data. Upon receiving theerror data and the vibration data (S3140), a decision is made at thecamera body 101 in step S3150 as to whether or not the two types of datahave been received, and the operation proceeds to step S3160 if thereception has been completed, whereas the operation returns to stepS3140 if the reception has not been completed, to continuously receivethe two types of data.

In step S3155, the point-image function is computed by using thereceived vibration data. In step S3160, the error data and thepoint-image function are saved into the image recording unit 120 incorrespondence to the photographed image. In step S3170, thephotographed image is saved into the image recording unit 120. It is tobe noted that unlike in step S3160, the error data are not saved in stepS3170. In step S3180, a command prompting the interchangeable lens 102to turn off the power is transmitted, and subsequently, the power to thecamera body 101 is turned off in step S3190, thereby ending theoperation.

Upon receiving the command prompting a power off (S4150), theinterchangeable lens 102 turns off the angular velocity sensor 10 andthe other circuits in step S4160, thereby ending the operation.

The image data, the point-image function having been computed by thepoint-image function computing unit 100 and the error data, which havebeen saved are transmitted to the image reproducing device 2 where thepoint-image function is corrected based upon the error data at thefunction correcting unit 240.

Subsequently, the image is restored at the image restoration computingunit 210. At this time, the image restoration is executed by using thepoint-image function having been corrected based upon the error data.The restored image obtained through this process will have undergoneblur correction executed by taking into consideration the positionalerror of the blur correction lens 70, thereby achieving a high blurcorrection effect in the restored image.

In the camera system achieved in the embodiment, which allows the use ofinterchangeable lenses, the drive error manifesting with regard to theblur correction lens 70 is detected and the image restoration isexecuted by taking into consideration the detected error. As a result,the optimal blur correction effect is achieved regardless of which of aplurality of interchangeable lenses with varying blur correction lensdrive characteristics is currently used.

The present invention is not limited to the embodiments described aboveand allows for numerous variations and modifications which are alsoconsidered to be equally contained within the scope of the presentinvention.

While the volume of data is reduced through the reducing processing inthe first and third embodiments, the present invention is not limited tothis example and the reference value may be computed and saved and theerror may be computed and saved without culling any data.

While the point-image function is computed based upon the referencevalue having been computed by using the output from the angular velocitysensor 10 or the vibration data and the point-image function thuscomputed is then corrected based upon the error information in theembodiments described above, the present invention is not limited tothis example and a function computed based upon both the reference valueand the vibration data may then be corrected.

Alternatively, the point-image function maybe computed by using theerror information in conjunction with either or both of the referencevalue or the vibration data, and the image restoration may then beexecuted without correcting the point-image function.

In addition, the point-image function may be computed by using the errorinformation alone without using the output from the angular velocitysensor 10. Since it is not necessary to store or transfer throughcommunication the reference value or the vibration data if thepoint-image function is computed based upon the error information alone,the work efficiency is improved and the length of time required for theoperation is reduced.

While the point-image function computing unit 100 is disposed at thecamera body 101 of the camera 1 in the embodiments, the presentinvention is not limited to this example and the point-image functioncomputing unit 100 may instead be installed at, for instance, the imagereproduction device 2. Likewise, while the function correcting unit 240is included in the image reproducing device 2 in the fifth embodiment,the present invention is not limited to this example and the functioncorrecting unit may instead be installed at the camera body 101 of thecamera 1. In other words, the information related to the controlposition error can be utilized by adopting any of various configurationsachieved through different combinations of functions achieved at thecamera body and the image reproduction device. Examples of suchcombinations are listed in Table 2 below. TABLE 2 IMAGE REPRODUCTION No.CAMERA . CAMERA BODY DEVICE REMARKS 1 POINT-IMAGE FUNCTION COMPUTINGUNIT FUNCTION CORRECTING UNIT IMAGE RESTORATION COMPUTING UNIT 2POINT-IMAGE FUNCTION COMPUTING UNIT (REFLECTS ERROR) IMAGE RESTORATIONCOMPUTING UNIT 3 POINT-IMAGE FUNCTION COMPUTING UNIT (EXECUTESCOMPUTATION BASED UPON ERROR ALONE) IMAGE RESTORATION COMPUTING UNIT 4POINT-IMAGE FUNCTION IMAGE RESTORATION CORRESPONDS COMPUTING UNITCOMPUTING UNIT TO THIRD FUNCTION CORRECTING EMBODIMENT UNIT 5POINT-IMAGE FUNCTION FUNCTION CORRESPONDS COMPUTING UNIT CORRECTING UNITTO FIFTH IMAGE RESTORATION EMBODIMENT COMPUTING UNIT 6 POINT-IMAGEFUNCTION IMAGE RESTORATION COMPUTING UNIT COMPUTING UNIT (REFLECTSERROR) 7 POINT-IMAGE FUNCTION IMAGE RESTORATION COMPUTING UNIT COMPUTINGUNIT (EXECUTES COMPUTATION BASED UPON ERROR ALONE) 8 POINT-IMAGEFUNCTION COMPUTING UNIT FUNCTION CORRECTING UNIT IMAGE RESTORATIONCOMPUTING UNIT 9 POINT - IMAGE FUNCTION COMPUTING UNIT (REFLECTS ERROR)IMAGE RESTORATION COMPUTING UNIT 10 POINT - IMAGE FUNCTION COMPUTINGUNIT (EXECUTES COMPUTATION BASED UPON ERROR ALONE) IMAGE RESTORATIONCOMPUTING UNIT

Combination No. 4 in Table 2 corresponds to the third embodiment andcombination No. 5 in Table 2 corresponds to the fifth embodiment. Theadvantages of the present invention can be realized with equaleffectiveness by adopting any of the combinations listed in Table 2.

In addition, in combination No. 3, 7 and 10 in Table 2, the referencevalue data and the vibration data do not need to be stored ortransferred through communication and thus, an improvement in the workefficiency and a reduction in the length of processing time are achievedas explained earlier.

The advantages that can be obtained in the embodiments of the presentinvention described in detail above are summarized below.

In the system which includes the control position error output unit thatoutputs a control position error indicating the difference between atarget drive position of the blur correction optical system controlledby the control unit and the actual drive position of the blur correctionoptical system output from the position detection unit and the imagerestoration computing unit that executes image processing on an imagecaptured by the image-capturing unit by taking into consideration thecontrol position error and thus corrects an image blur through imagerestoration, the residual image blur attributable to the drive controlerror of the blur correction optical system, too, can be corrected.Thus, even when the optical blur correction does not achieve idealresults, the image blur can be ultimately corrected with a high level ofreliability to achieve a highly effective blur correction at all times.

In the system having the function correcting unit, which corrects thepoint spread function by using the control position error, the imagerestoration computing unit executes the image restoration by using thepoint spread function having been corrected by the function correctingunit and thus, the residual image blur attributable to the drive controlerror of the blur correction optical system, too, can be corrected. As aresult, even when the optical blur correction does not achieve idealresults, the image blur can be corrected with a high level ofreliability to ultimately achieve a highly effective blur correction atall times.

In the system having the point spread function computing unit, whichcomputes the point spread function based upon the control positionerror, the point spread function reflecting the drive control error ofthe blur correction optical system is computed. Thus, the residual imageblur attributable to the drive control error of the blur correctionoptical system, too, can be corrected. Consequently, even when theoptical blur correction does not achieve ideal results, the image blurcan be corrected with a high level of reliability to ultimately achievea highly effective blur correction at all times.

INDUSTRIAL APPLICABILITY

While the explanation has been given above in reference to an example inwhich the present invention is adopted in a digital still camera used tocapture still images, the present invention may be adopted withcomparable effectiveness in digital cameras used to capture dynamicimages.

1-62. (canceled)
 63. A blur correction camera system comprising: avibration detection unit that detects a vibration and outputs avibration detection signal; a blur correction optical system that isdriven based upon the vibration detection signal and corrects an imageblur; an image-capturing unit that captures an image formed with aphotographic optical system that includes the blur correction opticalsystem; and an image restoration computing unit that corrects an imageblur by executing image restoration through image processing on an imagecaptured by the image-capturing unit.
 64. A blur correction camerasystem according to claim 63, further comprising: a point spreadfunction computing unit that computes a point spread function, wherein:the image restoration computing unit executes the image restoration byprocessing the image using the point spread function.
 65. A blurcorrection camera system according to claim 64, further comprising: areference value computing unit that computes a reference value for thevibration detection signal, wherein: the point spread function computingunit computes the point spread function based upon calculation resultsof the reference value computing unit.
 66. A blur correction camerasystem according to claim 65, comprising: a camera that comprises thevibration detection unit, the blur correction optical system, theimage-capturing unit, the point spread function computing unit, thereference value computing unit and an image recording unit that recordsan image; and an external device comprising the image restorationcomputing unit, that is a device independent of the camera and executesthe image restoration by using the image recorded by the image recordingunit and the point spread function input thereto.
 67. A blur correctioncamera comprising: a vibration detection unit that detects a vibrationand outputs a vibration detection signal; a reference value computingunit that computes a reference value for the vibration detection signal;a blur correction optical system that is driven based upon the referencevalue and the vibration detection signal and corrects an image blur; animage-capturing unit that captures an image formed by a photographicoptical system that includes the blur correction optical system; a pointspread function computing unit that computes a point spread functionneeded in an image restoration computation based upon the referencevalue; and an information volume reducing unit that reduces a volume ofinformation related to at least one of the reference value used in thecomputation of the point spread function and the computed point spreadfunction.
 68. A blur correction camera according to claim 67, wherein:the information volume reducing unit reduces the information volume byculling data related to at least one of the reference value and thecomputed point spread function.
 69. A blur correction camera accordingto claim 67, wherein: the information volume reducing unit reduces theinformation volume by ensuring that there will still be a large enoughvolume of information required for the image restoration computation.70. A blur correction camera system comprising: a vibration detectionunit that detects a vibration and outputs a vibration detection signal;an image-capturing unit that captures an image formed by a photographicoptical system that includes a blur correction optical system as a rawimage; an raw image saving unit that saves the raw image; an imagerestoration computing unit that allows parameters related to imageprocessing to be varied, executes image restoration through imageprocessing on the raw image by using the parameter and creates arestored image obtained by correcting an image blur; and a restorationresult saving unit that saves at least one of the parameters used in theimage processing executed at the image restoration computing unit andthe restored image in correspondence to the raw image.
 71. A blurcorrection camera system according to claim 70, further comprising: apoint spread function computing unit that computes a point spreadfunction, wherein: the image restoration computing unit executes theimage restoration by processing the image using the point spreadfunction; and the parameters include the point spread function.
 72. Ablur correction camera system according to claim 70, wherein: therestoration result saving unit is capable of saving at least one of aplurality of sets of parameters each corresponding to one of a pluralityof restored images and the plurality of restored images.
 73. A blurcorrection camera system according to claim 71, comprising: a camerathat comprises the vibration detection unit; the blur correction opticalsystem that is driven based upon the vibration detection signal andcorrects an image blur, the image-capturing unit, the point spreadfunction computing unit, a reference value computing unit that computesa reference value for the vibration detection signal and the raw imagesaving unit; and an external device comprising the image restorationcomputing unit and the restoration result saving unit, that is a deviceindependent of the camera and executes image restoration by using theraw image recorded at the raw image saving unit and the point spreadfunction input thereto.
 74. An image restoring device comprising: a datainput unit that receives raw image data and a point spread functionobtained when capturing the raw image data through at least one ofcommunication with an external device and a medium; an image restorationcomputing unit that allows a parameter related to image processing to bevaried, executes image restoration through executing image processing onthe raw image data using parameters that include the point spreadfunction and creates a restored image obtained by correcting an imageblur; and a restoration result saving unit that saves at least one ofthe parameters used in the image processing executed by the imagerestoration computing unit and the restored image in correspondence tothe raw image.
 75. A computer readable computer program productcontaining a blur correction control program, the control programcomprising the instructions of: receiving raw image data and a pointspread function obtained when capturing the raw image data; creating arestored image by executing image restoration so as to correct an imageblur through image processing executed on the raw image data usingvariable parameters related to the image processing that include thepoint spread function; and saving at least one of the parameters used inthe image processing during the image restoration computation and therestored image in correspondence to the raw image data.
 76. A computerprogram product according to claim 75, wherein: the computer programproduct is a recording medium on which the control program is recorded.77. A computer program product according to claim 75, wherein: thecomputer program product is a carrier wave on which the control programis embodied as a data signal.
 78. A blur correction camera comprising: avibration detection unit that detects a vibration and outputs avibration detection signal; an optical blur correction device thatcorrects an image blur by driving a blur correction optical system basedupon the vibration detection signal; a point spread function computingunit that computes a point spread function needed in image restorationin which the image blur is corrected through image processing; and animage restoration decision-making unit that makes a decision as towhether to enter an image restoration mode in which blur correction isexecuted through the image restoration or a preparatory operation forblur correction to be achieved through the image restoration isexecuted.
 79. A blur correction camera according to claim 78, wherein:the image restoration decision-making unit makes a decision as towhether to enter the image restoration mode based upon the vibrationdetection signal.
 80. A blur correction camera according to claim 78,wherein: the image restoration decision-making unit makes a decision asto whether to enter the image restoration mode based upon a shutterspeed.
 81. A blur correction camera according to claim 78, wherein: theimage restoration decision-making unit makes a decision as to whether toenter the image restoration mode based upon a focal length of aphotographic optical system.
 82. A blur correction camera according toclaim 78, wherein: the image restoration decision-making unit makes adecision as to whether to enter the image restoration mode based uponthe point spread function.
 83. A blur correction camera according toclaim 78, further comprising: a reporting device that reports a decisionmade by the image restoration decision-making unit that the imagerestoration mode should not be entered.
 84. A blur correction cameraaccording to claims 78, wherein: if the image restorationdecision-making unit determines that the image restoration mode shouldnot be entered, the image restoration mode is not entered.
 85. A blurcorrection camera according to claim 78, wherein: if the imagerestoration decision-making unit determines that the image restorationmode should not be entered, the point spread function is not saved.